2024-02-10T06:27:11.609 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-10T06:27:11.609 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156 --name yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose --description orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} -- /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/orig.config.yaml 2024-02-10T06:27:11.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T06:27:11.730 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156 branch: main description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} email: yweinste@redhat.com first_in_suite: false job_id: '7555156' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 1610 sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 targets: smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtfB6wEcBjgFpkXf65JhQSnrXfX7dab1uY+4Inh9Z3YvQ0l4YzMF9xaTxI+nVJrr3uvjyhLohrrJ9p05IN9NZ8= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJk03Y/wGdHEm0gebY0RdgrkzCNO+u2t7SRuau3alA2kUuzzhtq6HGfdkuXleBThex9QTSkevET9g3r4soWn2iY= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-09_19:44:16 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-10T06:27:11.730 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa; will attempt to use it 2024-02-10T06:27:11.731 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa/tasks 2024-02-10T06:27:11.731 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-10T06:27:11.733 INFO:teuthology.task.internal:Checking packages... 2024-02-10T06:27:11.757 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90' 2024-02-10T06:27:11.757 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-10T06:27:11.757 INFO:teuthology.packaging:ref: None 2024-02-10T06:27:11.757 INFO:teuthology.packaging:tag: None 2024-02-10T06:27:11.757 INFO:teuthology.packaging:branch: main 2024-02-10T06:27:11.757 INFO:teuthology.packaging:sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:27:11.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-10T06:27:12.026 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1361.gf8e4e331 2024-02-10T06:27:12.027 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-10T06:27:12.032 INFO:teuthology.task.internal:no buildpackages task found 2024-02-10T06:27:12.032 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-10T06:27:12.038 INFO:teuthology.task.internal:Saving configuration 2024-02-10T06:27:12.049 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-10T06:27:12.054 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T06:27:12.077 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:22:26.816383', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtfB6wEcBjgFpkXf65JhQSnrXfX7dab1uY+4Inh9Z3YvQ0l4YzMF9xaTxI+nVJrr3uvjyhLohrrJ9p05IN9NZ8='} 2024-02-10T06:27:12.100 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:22:26.818390', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJk03Y/wGdHEm0gebY0RdgrkzCNO+u2t7SRuau3alA2kUuzzhtq6HGfdkuXleBThex9QTSkevET9g3r4soWn2iY='} 2024-02-10T06:27:12.100 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-10T06:27:12.106 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-10T06:27:12.106 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-10T06:27:12.106 INFO:teuthology.run_tasks:Running task console_log... 2024-02-10T06:27:12.177 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbc47ec35e0>, signals=[15]) 2024-02-10T06:27:12.177 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-10T06:27:12.183 INFO:teuthology.task.internal:Opening connections... 2024-02-10T06:27:12.183 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-02-10T06:27:12.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:27:12.277 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-02-10T06:27:12.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:27:12.366 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-10T06:27:12.373 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-02-10T06:27:12.391 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-02-10T06:27:12.391 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T06:27:12.449 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-02-10T06:27:12.450 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-02-10T06:27:12.450 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T06:27:12.450 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-02-10T06:27:12.450 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T06:27:12.450 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T06:27:12.450 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T06:27:12.451 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-02-10T06:27:12.473 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-02-10T06:27:12.490 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-02-10T06:27:12.490 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-02-10T06:27:12.547 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T06:27:12.548 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T06:27:12.548 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-02-10T06:27:12.572 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-10T06:27:12.580 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-10T06:27:12.646 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-10T06:27:12.647 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-02-10T06:27:12.649 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-02-10T06:27:12.665 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-10T06:27:12.670 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-10T06:27:12.671 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-02-10T06:27:12.707 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-02-10T06:27:12.737 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-10T06:27:12.798 INFO:teuthology.run_tasks:Running task kernel... 2024-02-10T06:27:12.817 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-10T06:27:12.817 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-10T06:27:12.818 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T06:27:12.818 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T06:27:12.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:27:12.834 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-02-10T06:27:12.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:27:12.835 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-02-10T06:27:12.890 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-361.el9.x86_64 2024-02-10T06:27:12.890 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-361.el9.x86_64 2024-02-10T06:27:12.890 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-02-10T06:27:12.891 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-361.el9.x86_64 2024-02-10T06:27:12.891 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.14.0-361.el9.x86_64 2024-02-10T06:27:12.891 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2024-02-10T06:27:14.897 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-02-10T06:27:15.003 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-10T06:27:16.273 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-02-10T06:27:16.620 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-02-10T06:27:18.027 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-02-10T06:27:20.396 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-02-10T06:27:20.611 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-02-10T06:27:22.206 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-02-10T06:27:24.745 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 5.9 MB 00:00 2024-02-10T06:27:26.179 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-02-10T06:27:26.423 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 5.9 MB 00:00 2024-02-10T06:27:28.715 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-02-10T06:27:36.107 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 20 MB 00:07 2024-02-10T06:27:41.032 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-10T06:27:41.204 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 20 MB 00:14 2024-02-10T06:27:43.306 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T06:27:43.306 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T06:27:43.406 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:27:43.412 INFO:teuthology.orchestra.run.smithi174.stdout:Install 4 Packages 2024-02-10T06:27:43.413 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:27:43.413 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 99 M 2024-02-10T06:27:43.413 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 122 M 2024-02-10T06:27:43.413 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:27:43.891 INFO:teuthology.orchestra.run.smithi174.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-02-10T06:27:44.309 INFO:teuthology.orchestra.run.smithi174.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 23 MB/s | 21 MB 00:00 2024-02-10T06:27:44.984 INFO:teuthology.orchestra.run.smithi174.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 25 MB/s | 39 MB 00:01 2024-02-10T06:27:45.375 INFO:teuthology.orchestra.run.smithi174.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 23 MB/s | 33 MB 00:01 2024-02-10T06:27:45.378 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:27:45.379 INFO:teuthology.orchestra.run.smithi174.stdout:Total 51 MB/s | 99 MB 00:01 2024-02-10T06:27:45.379 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:27:45.746 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:27:45.746 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:27:46.103 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-10T06:27:46.780 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:27:46.780 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:27:48.427 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T06:27:48.427 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T06:27:48.514 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:27:48.524 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:27:48.527 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:27:48.527 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:27:48.528 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-02-10T06:27:48.529 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:27:48.529 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 99 M 2024-02-10T06:27:48.529 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 122 M 2024-02-10T06:27:48.529 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:27:48.947 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 14 MB/s | 6.0 MB 00:00 2024-02-10T06:27:49.191 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:27:49.480 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 22 MB/s | 21 MB 00:00 2024-02-10T06:27:49.849 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:27:50.000 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:27:50.480 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-10T06:27:50.780 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 18 MB/s | 33 MB 00:01 2024-02-10T06:27:50.784 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:27:50.784 INFO:teuthology.orchestra.run.smithi184.stdout:Total 44 MB/s | 99 MB 00:02 2024-02-10T06:27:50.784 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:27:51.129 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:27:51.129 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:27:52.168 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:27:52.169 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:27:52.703 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:27:52.885 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:27:54.583 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:27:55.241 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:27:55.373 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:27:57.610 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:27:57.814 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:27:58.151 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:27:58.327 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:28:02.555 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:03.012 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:28:03.114 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:07.834 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:31.821 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:31.821 INFO:teuthology.orchestra.run.smithi174.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T06:28:31.822 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:28:31.891 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:32.323 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:28:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:28:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:28:32.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:32.752 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:28:32.752 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:28:32.752 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T06:28:32.752 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T06:28:32.753 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T06:28:32.753 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T06:28:32.753 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:28:32.753 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:28:33.251 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:28:33.376 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:28:33.377 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:28:33.377 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T06:28:33.377 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T06:28:33.377 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T06:28:33.377 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-10T06:28:33.377 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T06:28:33.377 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:28:33.486 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:28:33.486 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:28:33.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2-tools 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2-tools 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.06 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 68.el9 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Group : Unspecified 2024-02-10T06:28:33.520 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 8264254 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : builder@centos.org 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Support tools for GRUB. 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T06:28:33.521 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2024-02-10T06:28:33.522 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:28:33.522 INFO:teuthology.orchestra.run.smithi174.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T06:28:33.522 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T06:28:33.522 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 5.14.0-419.el9.x86_64 2024-02-10T06:28:33.523 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T06:28:34.151 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2024-02-10T06:28:35.589 INFO:teuthology.orchestra.run.smithi174.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T06:28:35.616 INFO:teuthology.orchestra.run.smithi174.stderr:done 2024-02-10T06:28:35.618 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-02-10T06:28:35.635 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.ooHnzuM4wy 2024-02-10T06:28:35.635 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ooHnzuM4wy 2024-02-10T06:28:35.703 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.ooHnzuM4wy 2024-02-10T06:28:35.841 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.ooHnzuM4wy is 6KB 2024-02-10T06:28:35.890 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.ooHnzuM4wy 2024-02-10T06:28:35.904 DEBUG:teuthology.orchestra.run.smithi174:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T06:28:35.976 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T06:28:35.976 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T06:28:35.976 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T06:28:35.977 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T06:28:35.978 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T06:28:36.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2024-02-10T06:28:37.058 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:37.058 INFO:teuthology.orchestra.run.smithi184.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T06:28:37.058 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:28:37.128 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:37.525 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:37.525 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T06:28:37.525 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T06:28:37.526 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:28:37.986 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:28:38.496 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:28:38.635 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:28:38.635 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:28:38.635 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T06:28:38.635 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T06:28:38.635 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T06:28:38.635 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-10T06:28:38.635 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T06:28:38.635 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T06:28:38.744 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T06:28:38.744 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T06:28:38.744 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-02-10T06:28:38.778 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-02-10T06:28:38.778 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 68.el9 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8264254 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T06:28:38.779 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:28:38.780 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T06:28:38.781 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T06:28:38.781 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 5.14.0-419.el9.x86_64 2024-02-10T06:28:38.781 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T06:28:39.399 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-02-10T06:28:40.703 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T06:28:40.743 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-02-10T06:28:40.745 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-02-10T06:28:40.760 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.Y5BMRV1kSr 2024-02-10T06:28:40.760 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y5BMRV1kSr 2024-02-10T06:28:40.829 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.Y5BMRV1kSr 2024-02-10T06:28:40.963 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.Y5BMRV1kSr is 6KB 2024-02-10T06:28:41.012 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.Y5BMRV1kSr 2024-02-10T06:28:41.026 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T06:28:41.090 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T06:28:41.091 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T06:28:41.091 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T06:28:41.091 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T06:28:41.092 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T06:28:41.241 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-02-10T06:29:06.156 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-10T06:29:06.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-02-10T06:29:06.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:29:11.248 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-10T06:29:11.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-10T06:29:11.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:29:24.668 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-02-10T06:29:33.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-02-10T06:29:33.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:30:11.288 DEBUG:teuthology.orchestra.remote:timed out 2024-02-10T06:30:20.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-10T06:30:20.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:30:21.653 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-02-10T06:30:22.180 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-10T06:30:22.180 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-10T06:30:22.181 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-02-10T06:30:22.195 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-419.el9.x86_64 2024-02-10T06:30:22.195 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T06:30:22.195 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T06:30:22.195 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T06:30:23.196 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-10T06:30:23.196 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T06:30:23.305 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-02-10T06:30:23.353 DEBUG:teuthology.parallel:result is None 2024-02-10T06:30:33.676 DEBUG:teuthology.orchestra.remote:timed out 2024-02-10T06:30:45.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-02-10T06:30:45.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:30:46.068 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-02-10T06:30:46.450 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-02-10T06:30:46.450 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-10T06:30:46.450 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-02-10T06:30:46.466 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-419.el9.x86_64 2024-02-10T06:30:46.466 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T06:30:46.467 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T06:30:46.467 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T06:30:47.468 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-10T06:30:47.468 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T06:30:47.557 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2024-02-10T06:30:47.590 DEBUG:teuthology.parallel:result is None 2024-02-10T06:30:47.590 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-10T06:30:47.599 INFO:teuthology.task.internal:Creating test directory... 2024-02-10T06:30:47.599 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T06:30:47.601 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T06:30:47.621 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-10T06:30:47.628 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-10T06:30:47.634 INFO:teuthology.task.internal:Creating archive directory... 2024-02-10T06:30:47.635 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T06:30:47.660 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T06:30:47.692 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-10T06:30:47.699 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-10T06:30:47.699 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T06:30:47.725 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T06:30:47.764 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:30:47.765 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:30:47.777 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:30:47.780 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T06:30:47.782 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-10T06:30:47.789 INFO:teuthology.task.internal:Configuring sudo... 2024-02-10T06:30:47.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T06:30:47.822 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T06:30:47.852 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-10T06:30:47.862 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-10T06:30:47.862 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T06:30:47.891 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T06:30:47.909 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:30:47.991 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:30:48.040 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:30:48.122 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:30:48.165 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:30:48.165 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T06:30:48.231 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:30:48.277 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T06:30:48.336 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:30:48.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T06:30:48.422 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:30:48.422 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T06:30:48.491 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-02-10T06:30:48.493 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-02-10T06:30:48.547 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:30:48.584 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T06:30:49.024 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-10T06:30:49.032 INFO:teuthology.task.internal:Starting timer... 2024-02-10T06:30:49.032 INFO:teuthology.run_tasks:Running task pcp... 2024-02-10T06:30:49.041 INFO:teuthology.run_tasks:Running task selinux... 2024-02-10T06:30:49.049 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-10T06:30:49.050 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-02-10T06:30:49.097 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-02-10T06:30:49.099 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-02-10T06:30:49.159 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-02-10T06:30:49.161 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-10T06:30:49.161 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-02-10T06:30:49.191 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-02-10T06:30:49.191 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-02-10T06:30:49.214 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-02-10T06:30:49.214 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi174.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-02-10T06:30:49.215 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:30:49.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:30:49.260 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T06:30:49.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:30:49.287 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-10T06:30:49.287 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-02-10T06:30:49.335 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-02-10T06:30:49.366 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-10T06:30:49.376 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-10T06:30:49.392 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-10T06:30:49.393 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi174.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-10T06:35:06.809 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi174.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-02-10T06:35:06.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-02-10T06:35:06.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:35:06.907 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-02-10T06:35:06.987 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-02-10T06:35:06.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-10T06:35:06.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T06:35:07.087 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-02-10T06:35:07.163 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-02-10T06:35:07.164 INFO:teuthology.run_tasks:Running task clock... 2024-02-10T06:35:07.175 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-10T06:35:07.175 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T06:35:07.176 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:35:07.178 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T06:35:07.178 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T06:35:07.206 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T06:35:07.223 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T06:35:07.235 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T06:35:07.253 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T06:35:07.253 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-02-10T06:35:07.266 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-02-10T06:35:07.282 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-02-10T06:35:07.283 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T06:35:07.296 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-02-10T06:35:07.299 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T06:35:07.312 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T06:35:07.329 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T06:35:07.352 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:35:07.355 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:35:07.355 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-02-10T06:35:07.355 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.355 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.355 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.356 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.382 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-02-10T06:35:07.385 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T06:35:07.385 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-02-10T06:35:07.385 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.385 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.385 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.385 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T06:35:07.386 INFO:teuthology.run_tasks:Running task pexec... 2024-02-10T06:35:07.395 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-10T06:35:07.396 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi174.front.sepia.ceph.com 2024-02-10T06:35:07.396 DEBUG:teuthology.orchestra.run.smithi174:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T06:35:07.396 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi184.front.sepia.ceph.com 2024-02-10T06:35:07.397 DEBUG:teuthology.orchestra.run.smithi184:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T06:35:07.998 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:55 ago on Sat 10 Feb 2024 06:34:12 AM UTC. 2024-02-10T06:35:08.036 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:46 ago on Sat 10 Feb 2024 06:34:21 AM UTC. 2024-02-10T06:35:08.151 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:35:08.168 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:08.169 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 3.1 M 2024-02-10T06:35:08.169 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 9.8 M 2024-02-10T06:35:08.169 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:35:08.188 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:35:08.198 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:08.199 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.1 M 2024-02-10T06:35:08.199 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 9.8 M 2024-02-10T06:35:08.199 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:35:08.396 INFO:teuthology.orchestra.run.smithi174.stdout:runc-1.1.12-1.el9.x86_64.rpm 14 MB/s | 3.1 MB 00:00 2024-02-10T06:35:08.397 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:35:08.397 INFO:teuthology.orchestra.run.smithi174.stdout:Total 14 MB/s | 3.1 MB 00:00 2024-02-10T06:35:08.397 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:35:08.402 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:35:08.403 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:35:08.473 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:35:08.473 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:35:08.513 INFO:teuthology.orchestra.run.smithi184.stdout:runc-1.1.12-1.el9.x86_64.rpm 10 MB/s | 3.1 MB 00:00 2024-02-10T06:35:08.514 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:35:08.514 INFO:teuthology.orchestra.run.smithi184.stdout:Total 9.9 MB/s | 3.1 MB 00:00 2024-02-10T06:35:08.514 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:35:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:35:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:35:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:35:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:35:08.673 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:35:08.758 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:35:08.804 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:35:08.901 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:35:09.293 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:35:09.446 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:35:09.630 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:35:09.630 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:09.630 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:35:09.631 INFO:teuthology.orchestra.run.smithi174.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T06:35:09.631 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:09.631 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:35:09.773 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T06:35:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:09.774 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:35:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T06:35:09.774 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:09.774 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:35:09.868 DEBUG:teuthology.parallel:result is None 2024-02-10T06:35:10.048 DEBUG:teuthology.parallel:result is None 2024-02-10T06:35:10.049 INFO:teuthology.run_tasks:Running task install... 2024-02-10T06:35:10.057 DEBUG:teuthology.task.install:project ceph 2024-02-10T06:35:10.057 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'}, 'extra_packages': ['cephadm']} 2024-02-10T06:35:10.057 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'} 2024-02-10T06:35:10.057 INFO:teuthology.task.install:Using flavor: default 2024-02-10T06:35:10.063 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-10T06:35:10.063 INFO:teuthology.task.install:extra packages: [] 2024-02-10T06:35:10.063 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'tag': None, 'wait_for_package': False} 2024-02-10T06:35:10.064 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:35:10.066 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'tag': None, 'wait_for_package': False} 2024-02-10T06:35:10.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:35:10.229 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/ 2024-02-10T06:35:10.230 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1361.g290212a1 2024-02-10T06:35:10.251 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/ 2024-02-10T06:35:10.251 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1361.g290212a1 2024-02-10T06:35:10.355 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-10T06:35:10.355 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:35:10.355 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-10T06:35:10.373 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-10T06:35:10.373 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:35:10.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-10T06:35:10.383 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-10T06:35:10.383 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-10T06:35:10.411 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-10T06:35:10.412 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-10T06:35:10.450 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-10T06:35:10.479 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-10T06:35:10.530 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-10T06:35:10.559 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-10T06:35:10.597 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-02-10T06:35:10.598 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-02-10T06:35:10.625 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-02-10T06:35:10.627 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-02-10T06:35:10.862 INFO:teuthology.orchestra.run.smithi184.stdout:74 files removed 2024-02-10T06:35:10.867 INFO:teuthology.orchestra.run.smithi174.stdout:74 files removed 2024-02-10T06:35:10.889 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw 2024-02-10T06:35:10.894 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw 2024-02-10T06:35:11.595 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 211 kB/s | 82 kB 00:00 2024-02-10T06:35:11.602 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 209 kB/s | 82 kB 00:00 2024-02-10T06:35:11.929 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 35 kB/s | 10 kB 00:00 2024-02-10T06:35:11.969 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-02-10T06:35:12.263 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-02-10T06:35:12.311 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-02-10T06:35:12.682 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-02-10T06:35:12.822 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-02-10T06:35:14.107 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-02-10T06:35:14.322 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-02-10T06:35:14.948 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-02-10T06:35:17.071 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-02-10T06:35:18.851 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-02-10T06:35:20.222 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-02-10T06:35:24.581 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 5.9 MB 00:00 2024-02-10T06:35:24.841 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 5.9 MB 00:02 2024-02-10T06:35:26.315 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-02-10T06:35:26.392 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 15 kB 00:00 2024-02-10T06:35:27.791 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2024-02-10T06:35:29.577 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 20 MB 00:03 2024-02-10T06:35:32.799 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-02-10T06:35:34.460 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-02-10T06:35:35.478 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 9.7 M 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-02-10T06:35:35.480 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.4 M 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.3 M 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 20 M 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.0.0-1361.g290212a1.el9 ceph 25 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 743 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-10T06:35:35.481 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 499 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.1 M 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: x86_64 2:19.0.0-1361.g290212a1.el9 ceph 45 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 131 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.0.0-1361.g290212a1.el9 ceph 161 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.0.0-1361.g290212a1.el9 ceph 319 k 2024-02-10T06:35:35.482 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 298 k 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 99 k 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout:Install 30 Packages 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 2 Packages 2024-02-10T06:35:35.483 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:35.484 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 58 M 2024-02-10T06:35:35.484 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:35:36.883 INFO:teuthology.orchestra.run.smithi174.stdout:(1/32): ceph-base-19.0.0-1361.g290212a1.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-02-10T06:35:36.959 INFO:teuthology.orchestra.run.smithi174.stdout:(2/32): ceph-selinux-19.0.0-1361.g290212a1.el9. 335 kB/s | 25 kB 00:00 2024-02-10T06:35:37.059 INFO:teuthology.orchestra.run.smithi174.stdout:(3/32): libcephfs2-19.0.0-1361.g290212a1.el9.x8 7.3 MB/s | 743 kB 00:00 2024-02-10T06:35:37.097 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 9.7 M 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.4 M 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T06:35:37.100 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.3 M 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 20 M 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-1361.g290212a1.el9 ceph 25 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 743 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 499 k 2024-02-10T06:35:37.101 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.1 M 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: x86_64 2:19.0.0-1361.g290212a1.el9 ceph 45 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 131 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-1361.g290212a1.el9 ceph 161 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-10T06:35:37.102 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-1361.g290212a1.el9 ceph 319 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 298 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 99 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout:Install 30 Packages 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2024-02-10T06:35:37.103 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:37.104 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 58 M 2024-02-10T06:35:37.104 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:35:37.192 INFO:teuthology.orchestra.run.smithi174.stdout:(4/32): libradosstriper1-19.0.0-1361.g290212a1. 3.7 MB/s | 499 kB 00:00 2024-02-10T06:35:37.343 INFO:teuthology.orchestra.run.smithi174.stdout:(5/32): ceph-radosgw-19.0.0-1361.g290212a1.el9. 6.5 MB/s | 9.7 MB 00:01 2024-02-10T06:35:37.418 INFO:teuthology.orchestra.run.smithi174.stdout:(6/32): python3-ceph-argparse-19.0.0-1361.g2902 603 kB/s | 45 kB 00:00 2024-02-10T06:35:37.485 INFO:teuthology.orchestra.run.smithi174.stdout:(7/32): python3-ceph-common-19.0.0-1361.g290212 1.9 MB/s | 131 kB 00:00 2024-02-10T06:35:37.568 INFO:teuthology.orchestra.run.smithi174.stdout:(8/32): python3-cephfs-19.0.0-1361.g290212a1.el 1.9 MB/s | 161 kB 00:00 2024-02-10T06:35:37.660 INFO:teuthology.orchestra.run.smithi174.stdout:(9/32): python3-rados-19.0.0-1361.g290212a1.el9 3.4 MB/s | 319 kB 00:00 2024-02-10T06:35:37.761 INFO:teuthology.orchestra.run.smithi174.stdout:(10/32): python3-rbd-19.0.0-1361.g290212a1.el9. 2.9 MB/s | 298 kB 00:00 2024-02-10T06:35:37.819 INFO:teuthology.orchestra.run.smithi174.stdout:(11/32): librgw2-19.0.0-1361.g290212a1.el9.x86_ 8.1 MB/s | 5.1 MB 00:00 2024-02-10T06:35:37.844 INFO:teuthology.orchestra.run.smithi174.stdout:(12/32): python3-rgw-19.0.0-1361.g290212a1.el9. 1.2 MB/s | 99 kB 00:00 2024-02-10T06:35:37.962 INFO:teuthology.orchestra.run.smithi174.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-02-10T06:35:37.987 INFO:teuthology.orchestra.run.smithi174.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-02-10T06:35:38.012 INFO:teuthology.orchestra.run.smithi174.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-02-10T06:35:38.062 INFO:teuthology.orchestra.run.smithi174.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-02-10T06:35:38.087 INFO:teuthology.orchestra.run.smithi174.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-10T06:35:38.104 INFO:teuthology.orchestra.run.smithi174.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 993 kB/s | 41 kB 00:00 2024-02-10T06:35:38.163 INFO:teuthology.orchestra.run.smithi174.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 215 kB 00:00 2024-02-10T06:35:38.180 INFO:teuthology.orchestra.run.smithi174.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 451 kB/s | 34 kB 00:00 2024-02-10T06:35:38.197 INFO:teuthology.orchestra.run.smithi174.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-02-10T06:35:38.381 INFO:teuthology.orchestra.run.smithi174.stdout:(22/32): ceph-common-19.0.0-1361.g290212a1.el9. 7.8 MB/s | 20 MB 00:02 2024-02-10T06:35:38.681 INFO:teuthology.orchestra.run.smithi174.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 87 kB/s | 26 kB 00:00 2024-02-10T06:35:38.898 INFO:teuthology.orchestra.run.smithi174.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 224 kB/s | 49 kB 00:00 2024-02-10T06:35:39.106 INFO:teuthology.orchestra.run.smithi174.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-02-10T06:35:39.161 INFO:teuthology.orchestra.run.smithi184.stdout:(1/32): ceph-base-19.0.0-1361.g290212a1.el9.x86 2.8 MB/s | 5.3 MB 00:01 2024-02-10T06:35:39.294 INFO:teuthology.orchestra.run.smithi184.stdout:(2/32): ceph-radosgw-19.0.0-1361.g290212a1.el9. 4.8 MB/s | 9.7 MB 00:02 2024-02-10T06:35:39.311 INFO:teuthology.orchestra.run.smithi184.stdout:(3/32): ceph-selinux-19.0.0-1361.g290212a1.el9. 167 kB/s | 25 kB 00:00 2024-02-10T06:35:39.315 INFO:teuthology.orchestra.run.smithi174.stdout:(26/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 272 kB/s | 309 kB 00:01 2024-02-10T06:35:39.404 INFO:teuthology.orchestra.run.smithi184.stdout:(4/32): libcephfs2-19.0.0-1361.g290212a1.el9.x8 6.7 MB/s | 743 kB 00:00 2024-02-10T06:35:39.462 INFO:teuthology.orchestra.run.smithi184.stdout:(5/32): libradosstriper1-19.0.0-1361.g290212a1. 3.2 MB/s | 499 kB 00:00 2024-02-10T06:35:39.537 INFO:teuthology.orchestra.run.smithi184.stdout:(6/32): python3-ceph-argparse-19.0.0-1361.g2902 600 kB/s | 45 kB 00:00 2024-02-10T06:35:39.604 INFO:teuthology.orchestra.run.smithi184.stdout:(7/32): python3-ceph-common-19.0.0-1361.g290212 1.9 MB/s | 131 kB 00:00 2024-02-10T06:35:39.679 INFO:teuthology.orchestra.run.smithi184.stdout:(8/32): python3-cephfs-19.0.0-1361.g290212a1.el 2.1 MB/s | 161 kB 00:00 2024-02-10T06:35:39.698 INFO:teuthology.orchestra.run.smithi174.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 500 kB/s | 192 kB 00:00 2024-02-10T06:35:39.838 INFO:teuthology.orchestra.run.smithi184.stdout:(9/32): python3-rados-19.0.0-1361.g290212a1.el9 2.0 MB/s | 319 kB 00:00 2024-02-10T06:35:39.905 INFO:teuthology.orchestra.run.smithi184.stdout:(10/32): librgw2-19.0.0-1361.g290212a1.el9.x86_ 10 MB/s | 5.1 MB 00:00 2024-02-10T06:35:39.997 INFO:teuthology.orchestra.run.smithi184.stdout:(11/32): python3-rbd-19.0.0-1361.g290212a1.el9. 1.8 MB/s | 298 kB 00:00 2024-02-10T06:35:40.022 INFO:teuthology.orchestra.run.smithi184.stdout:(12/32): python3-rgw-19.0.0-1361.g290212a1.el9. 849 kB/s | 99 kB 00:00 2024-02-10T06:35:40.164 INFO:teuthology.orchestra.run.smithi184.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 634 kB/s | 106 kB 00:00 2024-02-10T06:35:40.190 INFO:teuthology.orchestra.run.smithi184.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 953 kB/s | 159 kB 00:00 2024-02-10T06:35:40.208 INFO:teuthology.orchestra.run.smithi184.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-10T06:35:40.257 INFO:teuthology.orchestra.run.smithi184.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-02-10T06:35:40.282 INFO:teuthology.orchestra.run.smithi184.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-10T06:35:40.299 INFO:teuthology.orchestra.run.smithi184.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 992 kB/s | 41 kB 00:00 2024-02-10T06:35:40.324 INFO:teuthology.orchestra.run.smithi184.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 5.0 MB/s | 215 kB 00:00 2024-02-10T06:35:40.341 INFO:teuthology.orchestra.run.smithi184.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 811 kB/s | 34 kB 00:00 2024-02-10T06:35:40.358 INFO:teuthology.orchestra.run.smithi184.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-02-10T06:35:40.559 INFO:teuthology.orchestra.run.smithi184.stdout:(22/32): ceph-common-19.0.0-1361.g290212a1.el9. 6.0 MB/s | 20 MB 00:03 2024-02-10T06:35:40.625 INFO:teuthology.orchestra.run.smithi184.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-02-10T06:35:40.667 INFO:teuthology.orchestra.run.smithi184.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 948 kB/s | 309 kB 00:00 2024-02-10T06:35:40.684 INFO:teuthology.orchestra.run.smithi184.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-02-10T06:35:40.710 INFO:teuthology.orchestra.run.smithi184.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-02-10T06:35:40.757 INFO:teuthology.orchestra.run.smithi174.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 508 kB/s | 838 kB 00:01 2024-02-10T06:35:40.768 INFO:teuthology.orchestra.run.smithi184.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-02-10T06:35:40.835 INFO:teuthology.orchestra.run.smithi184.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 838 kB 00:00 2024-02-10T06:35:41.027 INFO:teuthology.orchestra.run.smithi184.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-02-10T06:35:41.111 INFO:teuthology.orchestra.run.smithi184.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-02-10T06:35:41.394 INFO:teuthology.orchestra.run.smithi184.stdout:(31/32): librbd1-19.0.0-1361.g290212a1.el9.x86_ 8.5 MB/s | 3.1 MB 00:00 2024-02-10T06:35:41.432 INFO:teuthology.orchestra.run.smithi174.stdout:(29/32): librados2-19.0.0-1361.g290212a1.el9.x8 5.0 MB/s | 3.4 MB 00:00 2024-02-10T06:35:41.511 INFO:teuthology.orchestra.run.smithi184.stdout:(32/32): librados2-19.0.0-1361.g290212a1.el9.x8 5.0 MB/s | 3.4 MB 00:00 2024-02-10T06:35:41.515 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:35:41.515 INFO:teuthology.orchestra.run.smithi184.stdout:Total 13 MB/s | 58 MB 00:04 2024-02-10T06:35:41.515 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:35:41.550 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:35:41.550 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:35:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:(30/32): librbd1-19.0.0-1361.g290212a1.el9.x86_ 6.6 MB/s | 3.1 MB 00:00 2024-02-10T06:35:42.023 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:35:42.024 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:35:42.399 INFO:teuthology.orchestra.run.smithi174.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 600 kB/s | 1.6 MB 00:02 2024-02-10T06:35:43.500 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:35:43.595 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-10T06:35:43.625 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-10T06:35:43.650 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-10T06:35:43.681 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T06:35:43.697 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T06:35:43.853 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T06:35:43.911 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T06:35:44.228 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-10T06:35:44.293 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T06:35:44.384 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T06:35:44.480 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 8/34 2024-02-10T06:35:44.512 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T06:35:44.574 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T06:35:44.603 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-10T06:35:44.638 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 11/34 2024-02-10T06:35:44.706 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T06:35:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T06:35:44.776 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T06:35:44.804 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-10T06:35:44.907 INFO:teuthology.orchestra.run.smithi174.stdout:(32/32): libarrow-9.0.0-10.el9.x86_64.rpm 674 kB/s | 4.4 MB 00:06 2024-02-10T06:35:44.911 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-10T06:35:44.912 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:35:44.912 INFO:teuthology.orchestra.run.smithi174.stdout:Total 6.1 MB/s | 58 MB 00:09 2024-02-10T06:35:44.913 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:35:44.948 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-10T06:35:44.950 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:35:44.951 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:35:44.987 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-10T06:35:45.012 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-10T06:35:45.041 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-10T06:35:45.073 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-10T06:35:45.391 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-10T06:35:45.420 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T06:35:45.427 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:35:45.427 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:35:45.510 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T06:35:45.542 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 23/34 2024-02-10T06:35:45.587 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-10T06:35:46.040 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-10T06:35:46.073 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T06:35:46.144 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T06:35:46.214 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 27/34 2024-02-10T06:35:46.242 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 28/34 2024-02-10T06:35:46.908 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:35:46.989 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-10T06:35:47.029 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-10T06:35:47.054 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-10T06:35:47.085 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T06:35:47.101 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T06:35:47.300 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T06:35:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T06:35:47.687 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-10T06:35:47.747 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T06:35:47.830 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T06:35:47.921 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 8/34 2024-02-10T06:35:47.949 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T06:35:48.011 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T06:35:48.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-10T06:35:48.085 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 11/34 2024-02-10T06:35:48.153 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T06:35:48.176 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T06:35:48.221 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T06:35:48.249 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-10T06:35:48.353 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T06:35:48.359 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-10T06:35:48.397 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T06:35:48.409 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-10T06:35:48.449 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-10T06:35:48.482 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-10T06:35:48.511 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-10T06:35:48.544 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-10T06:35:48.865 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-10T06:35:48.890 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T06:35:48.954 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T06:35:48.979 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T06:35:49.011 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 23/34 2024-02-10T06:35:49.045 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-10T06:35:49.084 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T06:35:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T06:35:49.160 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-10T06:35:49.160 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-10T06:35:49.160 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:49.202 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T06:35:49.502 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-10T06:35:49.534 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T06:35:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T06:35:49.689 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 27/34 2024-02-10T06:35:49.719 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 28/34 2024-02-10T06:35:51.912 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T06:35:51.957 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T06:35:52.505 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T06:35:52.543 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T06:35:52.633 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T06:35:52.633 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-10T06:35:52.634 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-10T06:35:52.634 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:35:52.678 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-02-10T06:35:58.813 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-02-10T06:35:58.814 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:59.720 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T06:35:59.744 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T06:35:59.745 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:35:59.745 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-10T06:35:59.745 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T06:35:59.745 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T06:35:59.745 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:35:59.747 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T06:35:59.831 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T06:35:59.831 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T06:36:00.953 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T06:36:00.953 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 1/34 2024-02-10T06:36:00.953 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 3/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 4/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 5/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 6/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 8/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 9/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 10/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 11/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 13/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-10T06:36:00.954 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-10T06:36:00.955 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-10T06:36:00.956 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-10T06:36:00.956 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-10T06:36:00.956 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-10T06:36:00.956 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T06:36:00.956 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-10T06:36:00.956 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 33/34 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.133 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.134 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:02.135 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-02-10T06:36:02.176 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:02.535 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-test 2024-02-10T06:36:03.082 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T06:36:03.107 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T06:36:03.107 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:03.107 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-10T06:36:03.107 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T06:36:03.107 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T06:36:03.107 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:03.108 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T06:36:03.124 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:29 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:03.193 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T06:36:03.193 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T06:36:03.284 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-1361.g290212a1.el9 ceph 47 M 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-1361.g290212a1.el9 ceph 163 k 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-10T06:36:03.285 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 48 M 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 204 M 2024-02-10T06:36:03.286 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:03.547 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-02-10T06:36:03.606 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-02-10T06:36:03.673 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 186 kB 00:00 2024-02-10T06:36:03.690 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): libcephsqlite-19.0.0-1361.g290212a1.el9. 405 kB/s | 163 kB 00:00 2024-02-10T06:36:03.740 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 1/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 3/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 4/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 5/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 6/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 8/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 9/34 2024-02-10T06:36:04.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 10/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 11/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 13/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-10T06:36:04.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-10T06:36:04.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-10T06:36:04.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-10T06:36:04.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-10T06:36:04.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-10T06:36:04.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-10T06:36:04.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T06:36:04.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-10T06:36:04.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 33/34 2024-02-10T06:36:05.485 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T06:36:05.485 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:05.485 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-02-10T06:36:05.485 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.485 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.486 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-10T06:36:05.487 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:05.488 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:05.849 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-test 2024-02-10T06:36:06.073 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): ceph-test-19.0.0-1361.g290212a1.el9.x86_ 17 MB/s | 47 MB 00:02 2024-02-10T06:36:06.077 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:06.077 INFO:teuthology.orchestra.run.smithi184.stdout:Total 17 MB/s | 48 MB 00:02 2024-02-10T06:36:06.077 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:06.097 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:06.097 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:06.426 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:34 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:06.495 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:06.495 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:06.617 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:06.618 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:06.618 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:36:06.618 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:06.619 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.0.0-1361.g290212a1.el9 ceph 47 M 2024-02-10T06:36:06.619 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:36:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-10T06:36:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.0.0-1361.g290212a1.el9 ceph 163 k 2024-02-10T06:36:06.619 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-10T06:36:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-10T06:36:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-10T06:36:06.620 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:06.620 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:06.620 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:06.621 INFO:teuthology.orchestra.run.smithi174.stdout:Install 6 Packages 2024-02-10T06:36:06.621 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:06.622 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 48 M 2024-02-10T06:36:06.622 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 204 M 2024-02-10T06:36:06.622 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:06.789 INFO:teuthology.orchestra.run.smithi174.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-02-10T06:36:06.831 INFO:teuthology.orchestra.run.smithi174.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-10T06:36:06.873 INFO:teuthology.orchestra.run.smithi174.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-10T06:36:06.923 INFO:teuthology.orchestra.run.smithi174.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-10T06:36:07.087 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:07.148 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-10T06:36:07.191 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-10T06:36:07.233 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-10T06:36:07.240 INFO:teuthology.orchestra.run.smithi174.stdout:(5/6): libcephsqlite-19.0.0-1361.g290212a1.el9. 263 kB/s | 163 kB 00:00 2024-02-10T06:36:07.282 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-10T06:36:07.310 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T06:36:10.274 INFO:teuthology.orchestra.run.smithi174.stdout:(6/6): ceph-test-19.0.0-1361.g290212a1.el9.x86_ 13 MB/s | 47 MB 00:03 2024-02-10T06:36:10.277 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:10.277 INFO:teuthology.orchestra.run.smithi174.stdout:Total 13 MB/s | 48 MB 00:03 2024-02-10T06:36:10.277 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:10.297 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:10.298 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:10.698 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:10.698 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:11.283 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:36:11.354 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-10T06:36:11.390 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-10T06:36:11.434 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-10T06:36:11.474 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-10T06:36:11.493 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T06:36:11.796 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T06:36:11.876 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T06:36:14.363 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T06:36:14.364 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 1/6 2024-02-10T06:36:14.364 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2/6 2024-02-10T06:36:14.364 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-10T06:36:14.364 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-10T06:36:14.364 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.6-16.el9.x86_64 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-10T06:36:14.794 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-10T06:36:14.795 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-10T06:36:14.795 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:14.795 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:15.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph 2024-02-10T06:36:15.626 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:41 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:15.788 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-1361.g290212a1.el9 ceph 6.6 k 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:36:15.791 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-1361.g290212a1.el9 ceph 2.1 M 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-1361.g290212a1.el9 ceph 1.5 M 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 247 k 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-1361.g290212a1.el9 ceph 4.7 M 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 17 M 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-10T06:36:15.792 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-10T06:36:15.793 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:15.794 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:15.795 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-02-10T06:36:15.795 INFO:teuthology.orchestra.run.smithi184.stdout:Install 36 Packages 2024-02-10T06:36:15.795 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:15.795 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 30 M 2024-02-10T06:36:15.795 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 107 M 2024-02-10T06:36:15.795 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:15.955 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T06:36:16.033 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T06:36:16.334 INFO:teuthology.orchestra.run.smithi184.stdout:(1/36): ceph-19.0.0-1361.g290212a1.el9.x86_64.r 18 kB/s | 6.6 kB 00:00 2024-02-10T06:36:17.042 INFO:teuthology.orchestra.run.smithi184.stdout:(2/36): ceph-mgr-19.0.0-1361.g290212a1.el9.x86_ 1.4 MB/s | 1.5 MB 00:01 2024-02-10T06:36:17.259 INFO:teuthology.orchestra.run.smithi184.stdout:(3/36): ceph-mds-19.0.0-1361.g290212a1.el9.x86_ 1.7 MB/s | 2.1 MB 00:01 2024-02-10T06:36:17.418 INFO:teuthology.orchestra.run.smithi184.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1361.g2902 1.5 MB/s | 247 kB 00:00 2024-02-10T06:36:17.535 INFO:teuthology.orchestra.run.smithi184.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-02-10T06:36:17.577 INFO:teuthology.orchestra.run.smithi184.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-10T06:36:17.610 INFO:teuthology.orchestra.run.smithi184.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 948 kB/s | 32 kB 00:00 2024-02-10T06:36:17.644 INFO:teuthology.orchestra.run.smithi184.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-02-10T06:36:17.677 INFO:teuthology.orchestra.run.smithi184.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-10T06:36:17.719 INFO:teuthology.orchestra.run.smithi184.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-02-10T06:36:17.786 INFO:teuthology.orchestra.run.smithi184.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-02-10T06:36:17.812 INFO:teuthology.orchestra.run.smithi184.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 4.0 MB/s | 103 kB 00:00 2024-02-10T06:36:17.845 INFO:teuthology.orchestra.run.smithi184.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-02-10T06:36:17.879 INFO:teuthology.orchestra.run.smithi184.stdout:(14/36): python3-requests-2.25.1-8.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2024-02-10T06:36:17.921 INFO:teuthology.orchestra.run.smithi184.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-02-10T06:36:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T06:36:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 1/6 2024-02-10T06:36:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2/6 2024-02-10T06:36:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-10T06:36:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-10T06:36:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-10T06:36:18.663 INFO:teuthology.orchestra.run.smithi184.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-02-10T06:36:18.777 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-10T06:36:18.777 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:18.777 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:18.777 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:18.777 INFO:teuthology.orchestra.run.smithi174.stdout: jq-1.6-16.el9.x86_64 2024-02-10T06:36:18.778 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:18.778 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-10T06:36:18.778 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-10T06:36:18.778 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-10T06:36:18.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:18.778 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:19.004 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph 2024-02-10T06:36:19.080 INFO:teuthology.orchestra.run.smithi184.stdout:(17/36): ceph-mon-19.0.0-1361.g290212a1.el9.x86 1.7 MB/s | 4.7 MB 00:02 2024-02-10T06:36:19.105 INFO:teuthology.orchestra.run.smithi184.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 342 kB/s | 151 kB 00:00 2024-02-10T06:36:19.280 INFO:teuthology.orchestra.run.smithi184.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 983 kB/s | 172 kB 00:00 2024-02-10T06:36:19.400 INFO:teuthology.orchestra.run.smithi184.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-02-10T06:36:19.456 INFO:teuthology.orchestra.run.smithi184.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.0 MB/s | 358 kB 00:00 2024-02-10T06:36:19.498 INFO:teuthology.orchestra.run.smithi184.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-02-10T06:36:19.607 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:47 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:19.673 INFO:teuthology.orchestra.run.smithi184.stdout:(23/36): ceph-osd-19.0.0-1361.g290212a1.el9.x86 6.4 MB/s | 17 MB 00:02 2024-02-10T06:36:19.690 INFO:teuthology.orchestra.run.smithi184.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 75 kB/s | 18 kB 00:00 2024-02-10T06:36:19.707 INFO:teuthology.orchestra.run.smithi184.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 111 kB/s | 23 kB 00:00 2024-02-10T06:36:19.788 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:19.790 INFO:teuthology.orchestra.run.smithi174.stdout:========================================================================================= 2024-02-10T06:36:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:36:19.790 INFO:teuthology.orchestra.run.smithi174.stdout:========================================================================================= 2024-02-10T06:36:19.790 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.0.0-1361.g290212a1.el9 ceph 6.6 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.0.0-1361.g290212a1.el9 ceph 2.1 M 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.0.0-1361.g290212a1.el9 ceph 1.5 M 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 247 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.0.0-1361.g290212a1.el9 ceph 4.7 M 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 17 M 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-10T06:36:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-10T06:36:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout:========================================================================================= 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout:Install 36 Packages 2024-02-10T06:36:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:19.794 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 30 M 2024-02-10T06:36:19.794 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 107 M 2024-02-10T06:36:19.794 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:19.799 INFO:teuthology.orchestra.run.smithi184.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-02-10T06:36:19.815 INFO:teuthology.orchestra.run.smithi184.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 427 kB/s | 46 kB 00:00 2024-02-10T06:36:19.899 INFO:teuthology.orchestra.run.smithi184.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 787 kB/s | 79 kB 00:00 2024-02-10T06:36:19.916 INFO:teuthology.orchestra.run.smithi184.stdout:(29/36): python3-jaraco-functools-3.5.0-2.el9.n 80 kB/s | 19 kB 00:00 2024-02-10T06:36:19.991 INFO:teuthology.orchestra.run.smithi184.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-02-10T06:36:20.017 INFO:teuthology.orchestra.run.smithi184.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-02-10T06:36:20.092 INFO:teuthology.orchestra.run.smithi184.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 512 kB/s | 90 kB 00:00 2024-02-10T06:36:20.109 INFO:teuthology.orchestra.run.smithi184.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 305 kB/s | 36 kB 00:00 2024-02-10T06:36:20.134 INFO:teuthology.orchestra.run.smithi184.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-02-10T06:36:20.183 INFO:teuthology.orchestra.run.smithi174.stdout:(1/36): ceph-19.0.0-1361.g290212a1.el9.x86_64.r 29 kB/s | 6.6 kB 00:00 2024-02-10T06:36:20.201 INFO:teuthology.orchestra.run.smithi184.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-02-10T06:36:20.368 INFO:teuthology.orchestra.run.smithi184.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.5 MB/s | 427 kB 00:00 2024-02-10T06:36:20.371 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:20.371 INFO:teuthology.orchestra.run.smithi184.stdout:Total 6.5 MB/s | 30 MB 00:04 2024-02-10T06:36:20.390 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:20.433 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:20.433 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:20.855 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:20.856 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:20.991 INFO:teuthology.orchestra.run.smithi174.stdout:(2/36): ceph-mgr-19.0.0-1361.g290212a1.el9.x86_ 1.4 MB/s | 1.5 MB 00:01 2024-02-10T06:36:21.466 INFO:teuthology.orchestra.run.smithi174.stdout:(3/36): ceph-mds-19.0.0-1361.g290212a1.el9.x86_ 1.4 MB/s | 2.1 MB 00:01 2024-02-10T06:36:21.625 INFO:teuthology.orchestra.run.smithi174.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1361.g2902 1.5 MB/s | 247 kB 00:00 2024-02-10T06:36:21.742 INFO:teuthology.orchestra.run.smithi174.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-02-10T06:36:21.784 INFO:teuthology.orchestra.run.smithi174.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-10T06:36:21.818 INFO:teuthology.orchestra.run.smithi174.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 946 kB/s | 32 kB 00:00 2024-02-10T06:36:21.851 INFO:teuthology.orchestra.run.smithi174.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-02-10T06:36:21.910 INFO:teuthology.orchestra.run.smithi174.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 663 kB/s | 39 kB 00:00 2024-02-10T06:36:21.952 INFO:teuthology.orchestra.run.smithi174.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-02-10T06:36:22.019 INFO:teuthology.orchestra.run.smithi174.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-02-10T06:36:22.052 INFO:teuthology.orchestra.run.smithi174.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-02-10T06:36:22.086 INFO:teuthology.orchestra.run.smithi174.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-02-10T06:36:22.120 INFO:teuthology.orchestra.run.smithi174.stdout:(14/36): python3-requests-2.25.1-8.el9.noarch.r 3.3 MB/s | 113 kB 00:00 2024-02-10T06:36:22.154 INFO:teuthology.orchestra.run.smithi174.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-02-10T06:36:22.166 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:22.229 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-10T06:36:22.245 INFO:teuthology.orchestra.run.smithi174.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-02-10T06:36:22.283 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-10T06:36:22.314 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-10T06:36:22.362 INFO:teuthology.orchestra.run.smithi174.stdout:(17/36): ceph-mon-19.0.0-1361.g290212a1.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-02-10T06:36:22.384 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-10T06:36:22.429 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-10T06:36:22.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-10T06:36:22.553 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-10T06:36:22.631 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-10T06:36:22.675 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-10T06:36:22.679 INFO:teuthology.orchestra.run.smithi174.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-02-10T06:36:22.744 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-10T06:36:22.771 INFO:teuthology.orchestra.run.smithi174.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 287 kB/s | 151 kB 00:00 2024-02-10T06:36:22.798 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-10T06:36:22.845 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-10T06:36:22.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-10T06:36:22.949 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-10T06:36:22.988 INFO:teuthology.orchestra.run.smithi174.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 557 kB/s | 172 kB 00:00 2024-02-10T06:36:23.088 INFO:teuthology.orchestra.run.smithi174.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-02-10T06:36:23.135 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-10T06:36:23.189 INFO:teuthology.orchestra.run.smithi174.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 177 kB/s | 18 kB 00:00 2024-02-10T06:36:23.214 INFO:teuthology.orchestra.run.smithi174.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 810 kB/s | 358 kB 00:00 2024-02-10T06:36:23.241 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-10T06:36:23.280 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-10T06:36:23.281 INFO:teuthology.orchestra.run.smithi174.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 253 kB/s | 23 kB 00:00 2024-02-10T06:36:23.314 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-10T06:36:23.315 INFO:teuthology.orchestra.run.smithi174.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-02-10T06:36:23.344 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-10T06:36:23.373 INFO:teuthology.orchestra.run.smithi174.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-02-10T06:36:23.415 INFO:teuthology.orchestra.run.smithi174.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 462 kB/s | 46 kB 00:00 2024-02-10T06:36:23.420 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-10T06:36:23.455 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T06:36:23.488 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-10T06:36:23.528 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-10T06:36:23.548 INFO:teuthology.orchestra.run.smithi174.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 450 kB/s | 79 kB 00:00 2024-02-10T06:36:23.563 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-10T06:36:23.598 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-10T06:36:23.686 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-10T06:36:23.720 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-10T06:36:23.800 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-10T06:36:23.815 INFO:teuthology.orchestra.run.smithi174.stdout:(29/36): ceph-osd-19.0.0-1361.g290212a1.el9.x86 5.9 MB/s | 17 MB 00:02 2024-02-10T06:36:23.841 INFO:teuthology.orchestra.run.smithi174.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 56 kB/s | 16 kB 00:00 2024-02-10T06:36:23.866 INFO:teuthology.orchestra.run.smithi174.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 604 kB/s | 272 kB 00:00 2024-02-10T06:36:23.936 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-10T06:36:23.966 INFO:teuthology.orchestra.run.smithi174.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 286 kB/s | 36 kB 00:00 2024-02-10T06:36:24.054 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T06:36:24.200 INFO:teuthology.orchestra.run.smithi174.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 688 kB/s | 230 kB 00:00 2024-02-10T06:36:24.213 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 31/36 2024-02-10T06:36:24.233 INFO:teuthology.orchestra.run.smithi174.stdout:(34/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 215 kB/s | 90 kB 00:00 2024-02-10T06:36:24.242 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T06:36:24.267 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T06:36:24.267 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:24.267 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-10T06:36:24.267 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T06:36:24.267 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T06:36:24.267 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:24.292 INFO:teuthology.orchestra.run.smithi174.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-02-10T06:36:24.551 INFO:teuthology.orchestra.run.smithi174.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 731 kB/s | 427 kB 00:00 2024-02-10T06:36:24.554 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:24.554 INFO:teuthology.orchestra.run.smithi174.stdout:Total 6.2 MB/s | 30 MB 00:04 2024-02-10T06:36:24.574 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:24.617 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:24.617 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:25.035 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:25.036 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:25.692 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T06:36:25.719 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T06:36:25.719 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:25.719 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-10T06:36:25.719 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T06:36:25.719 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T06:36:25.719 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:26.160 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T06:36:26.183 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T06:36:26.183 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:26.183 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-10T06:36:26.184 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T06:36:26.184 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T06:36:26.184 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:26.355 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:36:26.394 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T06:36:26.411 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-10T06:36:26.416 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T06:36:26.417 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:26.417 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-10T06:36:26.417 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T06:36:26.417 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T06:36:26.417 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:26.455 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-10T06:36:26.483 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T06:36:26.488 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-10T06:36:26.568 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-10T06:36:26.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-10T06:36:26.645 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-10T06:36:26.737 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-10T06:36:26.816 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-10T06:36:26.856 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-10T06:36:26.929 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-10T06:36:26.982 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-10T06:36:27.020 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-10T06:36:27.068 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-10T06:36:27.124 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-10T06:36:27.309 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-10T06:36:27.357 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-10T06:36:27.396 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-10T06:36:27.447 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-10T06:36:27.477 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-10T06:36:27.544 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-10T06:36:27.579 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T06:36:27.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-10T06:36:27.652 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-10T06:36:27.687 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-10T06:36:27.721 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-10T06:36:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-10T06:36:27.852 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-10T06:36:27.925 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-10T06:36:28.060 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-10T06:36:28.196 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 1/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 3/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 4/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 5/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 6/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-10T06:36:28.285 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-10T06:36:28.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-10T06:36:28.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-10T06:36:28.371 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 31/36 2024-02-10T06:36:28.398 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T06:36:28.424 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T06:36:28.424 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:28.424 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-10T06:36:28.424 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T06:36:28.424 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T06:36:28.424 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:29.258 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-10T06:36:29.258 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:29.258 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-10T06:36:29.259 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-10T06:36:29.260 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:29.261 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:29.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-base 2024-02-10T06:36:29.849 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T06:36:29.877 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T06:36:29.877 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:29.877 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-10T06:36:29.877 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T06:36:29.877 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T06:36:29.878 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:30.218 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:56 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:30.317 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T06:36:30.330 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:36:30.339 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T06:36:30.339 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:30.339 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-10T06:36:30.339 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T06:36:30.340 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T06:36:30.340 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:30.385 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:30.386 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:36:30.386 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:30.462 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install cephadm 2024-02-10T06:36:30.559 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T06:36:30.582 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T06:36:30.582 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:30.582 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-10T06:36:30.582 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T06:36:30.582 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T06:36:30.582 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:30.647 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T06:36:31.066 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:57 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:31.223 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:31.225 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:31.225 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:36:31.225 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:31.225 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:31.225 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 748 k 2024-02-10T06:36:31.225 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:31.226 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:31.226 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:31.226 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:36:31.226 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:31.226 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 748 k 2024-02-10T06:36:31.226 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 754 k 2024-02-10T06:36:31.227 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:31.668 INFO:teuthology.orchestra.run.smithi184.stdout:cephadm-19.0.0-1361.g290212a1.el9.noarch.rpm 1.6 MB/s | 748 kB 00:00 2024-02-10T06:36:31.668 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:31.669 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.6 MB/s | 748 kB 00:00 2024-02-10T06:36:31.669 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:31.674 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:31.674 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:31.680 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:31.680 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:31.754 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:32.126 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:32.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:32.447 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T06:36:32.447 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 1/36 2024-02-10T06:36:32.447 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2/36 2024-02-10T06:36:32.447 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 3/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 4/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 5/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 6/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-10T06:36:32.448 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-10T06:36:32.449 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-10T06:36:32.450 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-10T06:36:32.450 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T06:36:32.450 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-10T06:36:32.451 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-10T06:36:32.451 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-10T06:36:32.451 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-10T06:36:32.451 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-10T06:36:32.618 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:32.917 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:32.917 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:32.917 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:32.918 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:32.918 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:32.918 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:33.161 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-immutable-object-cache 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:33.299 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-10T06:36:33.300 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-10T06:36:33.301 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-10T06:36:33.302 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:33.302 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:33.612 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-base 2024-02-10T06:36:33.752 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:59 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:33.915 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1361.g290212a1.el9 ceph 145 k 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 145 k 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 436 k 2024-02-10T06:36:33.916 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:34.188 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:02 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:34.232 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-immutable-object-cache-19.0.0-1361.g290212 462 kB/s | 145 kB 00:00 2024-02-10T06:36:34.232 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:34.232 INFO:teuthology.orchestra.run.smithi184.stdout:Total 460 kB/s | 145 kB 00:00 2024-02-10T06:36:34.233 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:34.240 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:34.240 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:34.291 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:34.291 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:34.293 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:36:34.344 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:34.345 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:36:34.345 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:34.414 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install cephadm 2024-02-10T06:36:34.427 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:34.516 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T06:36:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T06:36:34.540 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:34.540 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-10T06:36:34.540 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:34.996 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:02 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:35.153 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:35.154 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:35.154 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:36:35.154 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:35.154 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:35.154 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 748 k 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 748 k 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 754 k 2024-02-10T06:36:35.155 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T06:36:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:35.513 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:35.513 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:35.513 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:35.588 INFO:teuthology.orchestra.run.smithi174.stdout:cephadm-19.0.0-1361.g290212a1.el9.noarch.rpm 1.7 MB/s | 748 kB 00:00 2024-02-10T06:36:35.588 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:35.588 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.7 MB/s | 748 kB 00:00 2024-02-10T06:36:35.589 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:35.594 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:35.594 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:35.599 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:35.599 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:35.676 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:36:35.744 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr 2024-02-10T06:36:36.103 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:36.185 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:36.337 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:02 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:36.443 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:36:36.494 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:36.495 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:36:36.496 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:36.573 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-dashboard 2024-02-10T06:36:36.594 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:36.875 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T06:36:36.875 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:36.875 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:36.875 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:36.875 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:36.875 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:37.075 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-immutable-object-cache 2024-02-10T06:36:37.158 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:03 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:37.316 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:37.317 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 1.8 M 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 28 k 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 15 k 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-10T06:36:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-02-10T06:36:37.319 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:37.320 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 6.8 M 2024-02-10T06:36:37.320 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-02-10T06:36:37.320 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:37.673 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:05 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:37.811 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1361.g29 141 kB/s | 28 kB 00:00 2024-02-10T06:36:37.833 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:37.833 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:37.833 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2024-02-10T06:36:37.833 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1361.g290212a1.el9 ceph 145 k 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:37.834 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 145 k 2024-02-10T06:36:37.835 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 436 k 2024-02-10T06:36:37.835 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:37.836 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1361.g290 68 kB/s | 15 kB 00:00 2024-02-10T06:36:37.969 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.7 MB/s | 238 kB 00:00 2024-02-10T06:36:38.003 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.2 MB/s | 1.0 MB 00:00 2024-02-10T06:36:38.076 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-immutable-object-cache-19.0.0-1361.g290212 603 kB/s | 145 kB 00:00 2024-02-10T06:36:38.077 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:38.077 INFO:teuthology.orchestra.run.smithi174.stdout:Total 599 kB/s | 145 kB 00:00 2024-02-10T06:36:38.077 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:38.084 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:38.085 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:38.103 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1361.g290212a 3.6 MB/s | 1.8 MB 00:00 2024-02-10T06:36:38.136 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:38.136 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:38.137 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.1 MB/s | 863 kB 00:00 2024-02-10T06:36:38.187 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-02-10T06:36:38.274 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:36:38.312 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.1 MB/s | 144 kB 00:00 2024-02-10T06:36:38.338 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.6 MB/s | 548 kB 00:00 2024-02-10T06:36:38.343 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T06:36:38.363 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 614 kB/s | 31 kB 00:00 2024-02-10T06:36:38.367 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T06:36:38.367 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:36:38.368 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-10T06:36:38.368 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:38.405 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.6 MB/s | 2.0 MB 00:00 2024-02-10T06:36:38.430 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-02-10T06:36:38.433 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:38.433 INFO:teuthology.orchestra.run.smithi184.stdout:Total 6.1 MB/s | 6.8 MB 00:01 2024-02-10T06:36:38.451 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:38.467 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:38.468 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:38.633 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:38.633 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:39.176 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:39.247 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-10T06:36:39.282 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-10T06:36:39.328 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-10T06:36:39.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T06:36:39.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:39.356 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:39.356 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:36:39.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:39.356 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:39.359 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-10T06:36:39.454 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-10T06:36:39.574 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-10T06:36:39.631 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr 2024-02-10T06:36:39.660 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-10T06:36:39.705 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-10T06:36:39.734 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-10T06:36:39.764 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 10/12 2024-02-10T06:36:39.994 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 11/12 2024-02-10T06:36:40.024 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T06:36:40.218 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:08 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:40.324 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:36:40.376 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:40.377 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:36:40.377 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:40.447 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-dashboard 2024-02-10T06:36:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T06:36:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 1/12 2024-02-10T06:36:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 2/12 2024-02-10T06:36:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 3/12 2024-02-10T06:36:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-10T06:36:40.870 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-10T06:36:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-10T06:36:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-10T06:36:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-10T06:36:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-10T06:36:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-10T06:36:40.871 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-10T06:36:41.030 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:09 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:41.189 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:41.190 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-02-10T06:36:41.190 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:36:41.190 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-02-10T06:36:41.190 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 1.8 M 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 28 k 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 15 k 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-10T06:36:41.191 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout:Install 12 Packages 2024-02-10T06:36:41.192 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:41.193 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 6.8 M 2024-02-10T06:36:41.193 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 26 M 2024-02-10T06:36:41.193 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:41.255 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-10T06:36:41.256 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:41.257 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:41.529 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-10T06:36:41.554 INFO:teuthology.orchestra.run.smithi174.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1361.g290 76 kB/s | 15 kB 00:00 2024-02-10T06:36:41.571 INFO:teuthology.orchestra.run.smithi174.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1361.g29 130 kB/s | 28 kB 00:00 2024-02-10T06:36:41.696 INFO:teuthology.orchestra.run.smithi174.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 238 kB 00:00 2024-02-10T06:36:41.755 INFO:teuthology.orchestra.run.smithi174.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-02-10T06:36:41.813 INFO:teuthology.orchestra.run.smithi174.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1361.g290212a 3.8 MB/s | 1.8 MB 00:00 2024-02-10T06:36:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-02-10T06:36:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.4 MB/s | 548 kB 00:00 2024-02-10T06:36:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-02-10T06:36:41.998 INFO:teuthology.orchestra.run.smithi174.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 922 kB/s | 31 kB 00:00 2024-02-10T06:36:42.040 INFO:teuthology.orchestra.run.smithi174.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-02-10T06:36:42.081 INFO:teuthology.orchestra.run.smithi174.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 9.7 MB/s | 2.0 MB 00:00 2024-02-10T06:36:42.106 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:08 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:42.267 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 7.4 M 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-10T06:36:42.268 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-02-10T06:36:42.269 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:42.270 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 37 M 2024-02-10T06:36:42.270 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 209 M 2024-02-10T06:36:42.270 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:42.503 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 263 kB/s | 32 kB 00:00 2024-02-10T06:36:42.507 INFO:teuthology.orchestra.run.smithi174.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.0 MB/s | 863 kB 00:00 2024-02-10T06:36:42.510 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:42.511 INFO:teuthology.orchestra.run.smithi174.stdout:Total 5.2 MB/s | 6.8 MB 00:01 2024-02-10T06:36:42.529 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:42.537 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 535 kB/s | 17 kB 00:00 2024-02-10T06:36:42.546 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:42.546 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:42.595 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-02-10T06:36:42.620 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 420 kB/s | 34 kB 00:00 2024-02-10T06:36:42.713 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:42.714 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:42.805 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2024-02-10T06:36:42.871 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-02-10T06:36:42.896 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-02-10T06:36:43.080 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 802 kB 00:00 2024-02-10T06:36:43.230 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-02-10T06:36:43.235 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:36:43.256 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 194 kB 00:00 2024-02-10T06:36:43.316 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-10T06:36:43.350 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-10T06:36:43.397 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-10T06:36:43.428 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-10T06:36:43.456 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-02-10T06:36:43.518 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-10T06:36:43.645 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-10T06:36:43.698 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-1 5.6 MB/s | 7.4 MB 00:01 2024-02-10T06:36:43.700 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:43.700 INFO:teuthology.orchestra.run.smithi184.stdout:Total 26 MB/s | 37 MB 00:01 2024-02-10T06:36:43.712 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:43.736 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:43.736 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:43.738 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-10T06:36:43.782 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-10T06:36:43.810 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-10T06:36:43.840 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 10/12 2024-02-10T06:36:44.084 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:44.085 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:44.106 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 11/12 2024-02-10T06:36:44.134 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T06:36:44.787 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:44.862 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-10T06:36:44.893 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-10T06:36:44.902 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T06:36:44.902 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 1/12 2024-02-10T06:36:44.902 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 2/12 2024-02-10T06:36:44.902 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 3/12 2024-02-10T06:36:44.902 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-10T06:36:44.903 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-10T06:36:44.985 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-10T06:36:45.022 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-10T06:36:45.191 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T06:36:45.221 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T06:36:45.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-10T06:36:45.273 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:45.273 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:45.273 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:45.274 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:45.296 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-10T06:36:45.520 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-10T06:36:45.934 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-10T06:36:46.039 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-10T06:36:46.098 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:14 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:46.257 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 7.4 M 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:36:46.258 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2024-02-10T06:36:46.259 INFO:teuthology.orchestra.run.smithi174.stdout:Install 12 Packages 2024-02-10T06:36:46.260 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:46.260 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 37 M 2024-02-10T06:36:46.260 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 209 M 2024-02-10T06:36:46.260 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:46.521 INFO:teuthology.orchestra.run.smithi174.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 228 kB/s | 32 kB 00:00 2024-02-10T06:36:46.546 INFO:teuthology.orchestra.run.smithi174.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 727 kB/s | 17 kB 00:00 2024-02-10T06:36:46.572 INFO:teuthology.orchestra.run.smithi174.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.4 MB/s | 34 kB 00:00 2024-02-10T06:36:46.639 INFO:teuthology.orchestra.run.smithi174.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-02-10T06:36:46.822 INFO:teuthology.orchestra.run.smithi174.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2024-02-10T06:36:46.897 INFO:teuthology.orchestra.run.smithi174.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-02-10T06:36:46.922 INFO:teuthology.orchestra.run.smithi174.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.2 MB/s | 217 kB 00:00 2024-02-10T06:36:47.081 INFO:teuthology.orchestra.run.smithi174.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 802 kB 00:00 2024-02-10T06:36:47.232 INFO:teuthology.orchestra.run.smithi174.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-02-10T06:36:47.256 INFO:teuthology.orchestra.run.smithi174.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 194 kB 00:00 2024-02-10T06:36:47.267 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-10T06:36:47.407 INFO:teuthology.orchestra.run.smithi174.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-02-10T06:36:47.682 INFO:teuthology.orchestra.run.smithi174.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-1 5.7 MB/s | 7.4 MB 00:01 2024-02-10T06:36:47.685 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:47.685 INFO:teuthology.orchestra.run.smithi174.stdout:Total 26 MB/s | 37 MB 00:01 2024-02-10T06:36:47.697 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:47.722 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:47.722 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:48.072 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:48.072 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:48.292 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-10T06:36:48.320 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T06:36:48.775 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:36:48.838 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-10T06:36:48.870 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-10T06:36:48.962 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-10T06:36:49.006 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-10T06:36:49.201 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T06:36:49.230 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T06:36:49.306 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-10T06:36:49.985 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-10T06:36:50.100 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-10T06:36:50.888 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 1/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-10T06:36:50.889 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-10T06:36:51.288 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-10T06:36:51.288 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:51.288 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:51.288 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:51.289 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:51.344 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-10T06:36:51.733 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-rook 2024-02-10T06:36:52.323 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:18 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:36:52.363 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-10T06:36:52.395 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T06:36:52.485 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 52 k 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:36:52.486 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:36:52.487 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-10T06:36:52.488 INFO:teuthology.orchestra.run.smithi184.stdout:Install 13 Packages 2024-02-10T06:36:52.488 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:52.488 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 2.0 M 2024-02-10T06:36:52.488 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-02-10T06:36:52.488 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:36:52.961 INFO:teuthology.orchestra.run.smithi184.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 147 kB/s | 18 kB 00:00 2024-02-10T06:36:52.978 INFO:teuthology.orchestra.run.smithi184.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 174 kB/s | 24 kB 00:00 2024-02-10T06:36:53.028 INFO:teuthology.orchestra.run.smithi184.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-02-10T06:36:53.054 INFO:teuthology.orchestra.run.smithi184.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-10T06:36:53.079 INFO:teuthology.orchestra.run.smithi184.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-10T06:36:53.104 INFO:teuthology.orchestra.run.smithi184.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 857 kB/s | 43 kB 00:00 2024-02-10T06:36:53.129 INFO:teuthology.orchestra.run.smithi184.stdout:(7/13): ceph-mgr-rook-19.0.0-1361.g290212a1.el9 179 kB/s | 52 kB 00:00 2024-02-10T06:36:53.405 INFO:teuthology.orchestra.run.smithi184.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 50 kB/s | 15 kB 00:00 2024-02-10T06:36:53.430 INFO:teuthology.orchestra.run.smithi184.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 92 kB/s | 32 kB 00:00 2024-02-10T06:36:53.714 INFO:teuthology.orchestra.run.smithi184.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 208 kB/s | 59 kB 00:00 2024-02-10T06:36:53.939 INFO:teuthology.orchestra.run.smithi184.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 251 kB/s | 203 kB 00:00 2024-02-10T06:36:53.989 INFO:teuthology.orchestra.run.smithi184.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 326 kB/s | 90 kB 00:00 2024-02-10T06:36:54.814 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T06:36:54.814 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 1/12 2024-02-10T06:36:54.814 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-10T06:36:54.815 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-10T06:36:55.253 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:55.254 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:36:55.670 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-rook 2024-02-10T06:36:55.689 INFO:teuthology.orchestra.run.smithi184.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 458 kB/s | 1.0 MB 00:02 2024-02-10T06:36:55.691 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:55.691 INFO:teuthology.orchestra.run.smithi184.stdout:Total 655 kB/s | 2.0 MB 00:03 2024-02-10T06:36:55.692 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:36:55.706 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:36:55.706 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:36:55.813 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:36:55.814 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:36:56.263 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:24 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:36:56.337 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:36:56.420 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:36:56.421 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 52 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-10T06:36:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout:Install 13 Packages 2024-02-10T06:36:56.423 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:36:56.424 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 2.0 M 2024-02-10T06:36:56.424 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 26 M 2024-02-10T06:36:56.424 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:36:56.502 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-10T06:36:56.565 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-10T06:36:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-10T06:36:56.658 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-10T06:36:56.700 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-10T06:36:56.728 INFO:teuthology.orchestra.run.smithi174.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 187 kB/s | 24 kB 00:00 2024-02-10T06:36:56.753 INFO:teuthology.orchestra.run.smithi174.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 116 kB/s | 18 kB 00:00 2024-02-10T06:36:56.793 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-10T06:36:56.803 INFO:teuthology.orchestra.run.smithi174.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-02-10T06:36:56.829 INFO:teuthology.orchestra.run.smithi174.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-10T06:36:56.854 INFO:teuthology.orchestra.run.smithi174.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-10T06:36:56.879 INFO:teuthology.orchestra.run.smithi174.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 856 kB/s | 43 kB 00:00 2024-02-10T06:36:56.901 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-10T06:36:56.904 INFO:teuthology.orchestra.run.smithi174.stdout:(7/13): ceph-mgr-rook-19.0.0-1361.g290212a1.el9 170 kB/s | 52 kB 00:00 2024-02-10T06:36:56.950 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-10T06:36:57.163 INFO:teuthology.orchestra.run.smithi174.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-02-10T06:36:57.188 INFO:teuthology.orchestra.run.smithi174.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 48 kB/s | 15 kB 00:00 2024-02-10T06:36:57.472 INFO:teuthology.orchestra.run.smithi174.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 208 kB/s | 59 kB 00:00 2024-02-10T06:36:57.673 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-10T06:36:57.723 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-10T06:36:57.730 INFO:teuthology.orchestra.run.smithi174.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 245 kB/s | 203 kB 00:00 2024-02-10T06:36:57.756 INFO:teuthology.orchestra.run.smithi174.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 316 kB/s | 90 kB 00:00 2024-02-10T06:36:57.757 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-10T06:36:57.806 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-10T06:36:57.835 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T06:36:58.631 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T06:36:58.631 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 1/13 2024-02-10T06:36:58.631 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-10T06:36:58.631 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-10T06:36:58.632 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-10T06:36:59.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-10T06:36:59.014 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-10T06:36:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-10T06:36:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-10T06:36:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-10T06:36:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-10T06:36:59.015 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:36:59.015 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:36:59.247 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-cephadm 2024-02-10T06:36:59.364 INFO:teuthology.orchestra.run.smithi174.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 475 kB/s | 1.0 MB 00:02 2024-02-10T06:36:59.366 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:36:59.367 INFO:teuthology.orchestra.run.smithi174.stdout:Total 713 kB/s | 2.0 MB 00:02 2024-02-10T06:36:59.367 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:36:59.381 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:36:59.382 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:36:59.487 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:36:59.498 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:36:59.843 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:25 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:00.014 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 138 k 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-10T06:37:00.015 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-02-10T06:37:00.016 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:00.017 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 6.8 M 2024-02-10T06:37:00.017 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 33 M 2024-02-10T06:37:00.017 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:00.031 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:00.194 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-10T06:37:00.240 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-10T06:37:00.295 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-10T06:37:00.316 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 227 kB 00:00 2024-02-10T06:37:00.332 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-10T06:37:00.373 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-10T06:37:00.466 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-10T06:37:00.474 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-02-10T06:37:00.500 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): ceph-mgr-cephadm-19.0.0-1361.g290212a1.e 402 kB/s | 138 kB 00:00 2024-02-10T06:37:00.567 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-10T06:37:00.575 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 2.1 MB/s | 548 kB 00:00 2024-02-10T06:37:00.600 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 460 kB/s | 58 kB 00:00 2024-02-10T06:37:00.625 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 407 kB/s | 51 kB 00:00 2024-02-10T06:37:00.625 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-10T06:37:00.627 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:00.627 INFO:teuthology.orchestra.run.smithi184.stdout:Total 11 MB/s | 6.8 MB 00:00 2024-02-10T06:37:00.627 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:00.636 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:00.637 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:00.728 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:00.728 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:01.018 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:01.173 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-10T06:37:01.227 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-10T06:37:01.350 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-10T06:37:01.397 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-10T06:37:01.430 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-10T06:37:01.479 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-10T06:37:01.501 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T06:37:01.749 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-10T06:37:01.837 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-10T06:37:01.899 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-10T06:37:01.922 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 1/13 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-10T06:37:02.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-10T06:37:02.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-10T06:37:02.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-10T06:37:02.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-10T06:37:02.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-10T06:37:02.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-10T06:37:02.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-10T06:37:02.706 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:02.707 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:02.725 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T06:37:02.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/6 2024-02-10T06:37:02.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-10T06:37:02.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-10T06:37:02.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-10T06:37:02.725 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-10T06:37:02.937 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-cephadm 2024-02-10T06:37:03.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:03.038 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:03.285 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-fuse 2024-02-10T06:37:03.524 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:31 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:03.686 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 138 k 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-10T06:37:03.687 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:03.688 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:03.688 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================== 2024-02-10T06:37:03.688 INFO:teuthology.orchestra.run.smithi174.stdout:Install 6 Packages 2024-02-10T06:37:03.688 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:03.689 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 6.8 M 2024-02-10T06:37:03.689 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 33 M 2024-02-10T06:37:03.689 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:03.874 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:29 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 885 k 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-10T06:37:04.035 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:04.036 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:04.036 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:04.036 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-02-10T06:37:04.036 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:04.036 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 964 k 2024-02-10T06:37:04.037 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 2.8 M 2024-02-10T06:37:04.037 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:04.162 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 627 kB/s | 79 kB 00:00 2024-02-10T06:37:04.217 INFO:teuthology.orchestra.run.smithi174.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 227 kB 00:00 2024-02-10T06:37:04.367 INFO:teuthology.orchestra.run.smithi174.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-02-10T06:37:04.370 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-fuse-19.0.0-1361.g290212a1.el9.x86_ 2.6 MB/s | 885 kB 00:00 2024-02-10T06:37:04.372 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:04.373 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.8 MB/s | 964 kB 00:00 2024-02-10T06:37:04.373 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:04.393 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:04.393 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:04.442 INFO:teuthology.orchestra.run.smithi174.stdout:(3/6): ceph-mgr-cephadm-19.0.0-1361.g290212a1.e 345 kB/s | 138 kB 00:00 2024-02-10T06:37:04.469 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:04.470 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:04.657 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:04.734 INFO:teuthology.orchestra.run.smithi174.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 157 kB/s | 58 kB 00:00 2024-02-10T06:37:04.805 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-10T06:37:04.809 INFO:teuthology.orchestra.run.smithi174.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 139 kB/s | 51 kB 00:00 2024-02-10T06:37:04.870 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T06:37:05.547 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T06:37:05.548 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/2 2024-02-10T06:37:05.784 INFO:teuthology.orchestra.run.smithi174.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 350 kB/s | 548 kB 00:01 2024-02-10T06:37:05.786 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:05.787 INFO:teuthology.orchestra.run.smithi174.stdout:Total 3.2 MB/s | 6.8 MB 00:02 2024-02-10T06:37:05.787 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:05.797 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:05.797 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-10T06:37:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:05.876 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-10T06:37:05.876 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:05.876 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:05.891 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:05.892 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:06.157 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-volume 2024-02-10T06:37:06.190 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:06.356 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-10T06:37:06.402 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-10T06:37:06.764 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:32 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:06.917 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-10T06:37:06.924 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:06.924 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:06.924 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-02-10T06:37:06.924 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:06.924 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 264 k 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-02-10T06:37:06.925 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:06.926 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 332 k 2024-02-10T06:37:06.926 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.5 M 2024-02-10T06:37:06.926 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:06.996 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-10T06:37:07.050 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 554 kB/s | 69 kB 00:00 2024-02-10T06:37:07.058 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-10T06:37:07.081 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T06:37:07.217 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-volume-19.0.0-1361.g290212a1.el9.no 907 kB/s | 264 kB 00:00 2024-02-10T06:37:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:07.218 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-02-10T06:37:07.219 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:07.227 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:07.227 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:07.267 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:07.267 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:07.419 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:07.599 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-10T06:37:07.636 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T06:37:07.651 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T06:37:07.651 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:37:07.651 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-10T06:37:07.651 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:07.850 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T06:37:07.851 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/6 2024-02-10T06:37:07.851 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-10T06:37:07.851 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-10T06:37:07.851 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-10T06:37:07.851 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-10T06:37:08.169 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:08.170 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:08.407 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-fuse 2024-02-10T06:37:08.496 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 1/2 2024-02-10T06:37:08.863 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-10T06:37:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:08.864 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:37:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-10T06:37:08.864 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:08.864 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:36 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:09.146 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 885 k 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout:Install 2 Packages 2024-02-10T06:37:09.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:09.148 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 964 k 2024-02-10T06:37:09.148 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.8 M 2024-02-10T06:37:09.148 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:09.180 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados-devel 2024-02-10T06:37:09.293 INFO:teuthology.orchestra.run.smithi174.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 543 kB/s | 79 kB 00:00 2024-02-10T06:37:09.510 INFO:teuthology.orchestra.run.smithi174.stdout:(2/2): ceph-fuse-19.0.0-1361.g290212a1.el9.x86_ 2.4 MB/s | 885 kB 00:00 2024-02-10T06:37:09.511 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:09.511 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.6 MB/s | 964 kB 00:00 2024-02-10T06:37:09.511 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:09.532 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:09.532 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:09.608 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:09.608 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:09.773 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:35 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:09.820 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:09.932 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:09.932 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 127 k 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 127 k 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 456 k 2024-02-10T06:37:09.933 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:09.961 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-10T06:37:10.034 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T06:37:10.156 INFO:teuthology.orchestra.run.smithi184.stdout:librados-devel-19.0.0-1361.g290212a1.el9.x86_64 571 kB/s | 127 kB 00:00 2024-02-10T06:37:10.156 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:10.156 INFO:teuthology.orchestra.run.smithi184.stdout:Total 567 kB/s | 127 kB 00:00 2024-02-10T06:37:10.156 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:10.162 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:10.162 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:10.218 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:10.218 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:10.420 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:10.529 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:10.707 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T06:37:10.707 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/2 2024-02-10T06:37:11.004 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:11.022 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-10T06:37:11.022 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:11.022 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:11.022 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-10T06:37:11.022 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:11.022 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:11.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:11.299 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:11.299 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:11.299 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:11.299 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:11.299 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:11.305 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-volume 2024-02-10T06:37:11.533 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs2 2024-02-10T06:37:11.908 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:39 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 264 k 2024-02-10T06:37:12.070 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout:Install 2 Packages 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 332 k 2024-02-10T06:37:12.071 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 1.5 M 2024-02-10T06:37:12.072 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:12.134 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:38 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:12.189 INFO:teuthology.orchestra.run.smithi174.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 589 kB/s | 69 kB 00:00 2024-02-10T06:37:12.241 INFO:teuthology.orchestra.run.smithi184.stdout:Package libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:12.292 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:12.293 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:12.293 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:12.331 INFO:teuthology.orchestra.run.smithi174.stdout:(2/2): ceph-volume-19.0.0-1361.g290212a1.el9.no 1.0 MB/s | 264 kB 00:00 2024-02-10T06:37:12.332 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:12.332 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.2 MB/s | 332 kB 00:00 2024-02-10T06:37:12.332 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:12.341 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:12.341 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:12.364 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs-devel 2024-02-10T06:37:12.380 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:12.380 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:12.531 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:12.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-10T06:37:12.758 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T06:37:12.774 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T06:37:12.774 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:37:12.774 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-10T06:37:12.775 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:12.949 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:38 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:13.108 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:13.108 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:13.108 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 31 k 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 31 k 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 140 k 2024-02-10T06:37:13.109 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:13.292 INFO:teuthology.orchestra.run.smithi184.stdout:libcephfs-devel-19.0.0-1361.g290212a1.el9.x86_6 171 kB/s | 31 kB 00:00 2024-02-10T06:37:13.292 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:13.293 INFO:teuthology.orchestra.run.smithi184.stdout:Total 169 kB/s | 31 kB 00:00 2024-02-10T06:37:13.293 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:13.296 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:13.296 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:13.333 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:13.334 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:13.531 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:13.563 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 1/2 2024-02-10T06:37:13.914 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-10T06:37:13.914 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:13.914 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:13.915 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T06:37:13.915 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-10T06:37:13.915 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:13.915 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:13.919 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:14.199 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:14.199 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:14.199 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:14.199 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:14.199 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:14.250 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados-devel 2024-02-10T06:37:14.372 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados2 2024-02-10T06:37:14.845 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:42 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:14.955 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:40 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 127 k 2024-02-10T06:37:15.007 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 127 k 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 456 k 2024-02-10T06:37:15.008 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:15.063 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:15.115 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:15.116 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:15.116 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:15.183 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librbd1 2024-02-10T06:37:15.252 INFO:teuthology.orchestra.run.smithi174.stdout:librados-devel-19.0.0-1361.g290212a1.el9.x86_64 521 kB/s | 127 kB 00:00 2024-02-10T06:37:15.252 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:15.253 INFO:teuthology.orchestra.run.smithi174.stdout:Total 518 kB/s | 127 kB 00:00 2024-02-10T06:37:15.253 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:15.258 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:15.258 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:15.316 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:15.316 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:15.514 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:15.625 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:15.762 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:41 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:15.870 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:15.923 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:15.924 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:15.924 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:15.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rados 2024-02-10T06:37:16.094 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:16.385 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:16.385 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:16.386 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:16.386 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:16.386 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:16.386 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:16.572 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:42 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:16.604 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs2 2024-02-10T06:37:16.678 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:16.728 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:16.730 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:16.730 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:16.801 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rgw 2024-02-10T06:37:17.206 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:45 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:17.316 INFO:teuthology.orchestra.run.smithi174.stdout:Package libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:17.368 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:17.369 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:17.370 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:17.384 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:43 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:17.425 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs-devel 2024-02-10T06:37:17.491 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:17.541 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:17.542 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:17.542 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:17.606 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-cephfs 2024-02-10T06:37:18.010 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:46 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:18.168 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:18.169 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:18.169 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2024-02-10T06:37:18.169 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:18.169 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:18.169 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 31 k 2024-02-10T06:37:18.169 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 31 k 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 140 k 2024-02-10T06:37:18.170 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:18.196 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:44 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:18.302 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:18.353 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:18.354 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:18.355 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:18.362 INFO:teuthology.orchestra.run.smithi174.stdout:libcephfs-devel-19.0.0-1361.g290212a1.el9.x86_6 162 kB/s | 31 kB 00:00 2024-02-10T06:37:18.363 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:18.363 INFO:teuthology.orchestra.run.smithi174.stdout:Total 161 kB/s | 31 kB 00:00 2024-02-10T06:37:18.363 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:18.367 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:18.367 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:18.390 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:18.390 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:18.422 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rbd 2024-02-10T06:37:18.480 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:18.594 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:19.007 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:19.011 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:45 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:19.117 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:19.169 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:19.170 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-02-10T06:37:19.170 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:19.242 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-fuse 2024-02-10T06:37:19.293 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:19.293 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:19.293 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:19.293 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:19.294 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:19.294 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:19.473 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados2 2024-02-10T06:37:19.828 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:45 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:19.986 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:19.987 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:19.987 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-02-10T06:37:19.987 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:19.987 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:19.987 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 85 k 2024-02-10T06:37:19.987 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 85 k 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 231 k 2024-02-10T06:37:19.988 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:20.052 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:48 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:20.159 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:20.211 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-fuse-19.0.0-1361.g290212a1.el9.x86_64.rpm 382 kB/s | 85 kB 00:00 2024-02-10T06:37:20.212 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:20.212 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:20.212 INFO:teuthology.orchestra.run.smithi184.stdout:Total 380 kB/s | 85 kB 00:00 2024-02-10T06:37:20.213 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:20.213 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:20.213 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:20.219 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:20.273 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:20.274 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:20.282 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librbd1 2024-02-10T06:37:20.416 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:20.510 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:20.869 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:48 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:20.909 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:20.976 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:21.029 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:21.030 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:21.030 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:21.103 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rados 2024-02-10T06:37:21.197 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:21.198 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:21.198 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:21.198 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:21.455 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-mirror 2024-02-10T06:37:21.695 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:49 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:21.802 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:21.853 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:21.854 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:21.854 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:21.929 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rgw 2024-02-10T06:37:22.053 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:48 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:22.217 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:22.218 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:37:22.219 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:22.219 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.1 M 2024-02-10T06:37:22.219 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 13 M 2024-02-10T06:37:22.219 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:22.517 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:50 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:22.623 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:22.656 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-mirror-19.0.0-1361.g290212a1.el9.x86_64.rpm 7.1 MB/s | 3.1 MB 00:00 2024-02-10T06:37:22.656 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:22.657 INFO:teuthology.orchestra.run.smithi184.stdout:Total 7.0 MB/s | 3.1 MB 00:00 2024-02-10T06:37:22.657 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:22.666 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:22.666 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:22.673 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:22.675 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:22.675 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:22.739 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:22.739 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:22.741 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-cephfs 2024-02-10T06:37:23.159 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:23.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:23.238 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:23.239 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:37:23.239 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-10T06:37:23.239 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T06:37:23.239 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T06:37:23.239 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:23.359 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:51 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:23.470 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:23.525 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:23.526 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:23.526 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:23.592 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rbd 2024-02-10T06:37:24.203 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:52 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:24.310 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T06:37:24.358 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:24.358 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:24.358 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:24.358 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:24.358 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:24.358 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:24.360 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:24.361 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-02-10T06:37:24.362 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:24.444 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-fuse 2024-02-10T06:37:24.603 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-nbd 2024-02-10T06:37:25.023 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:53 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:25.185 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:25.185 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 85 k 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 85 k 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 231 k 2024-02-10T06:37:25.186 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:25.206 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:51 ago on Sat 10 Feb 2024 06:35:34 AM UTC. 2024-02-10T06:37:25.367 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 171 k 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-02-10T06:37:25.368 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-02-10T06:37:25.369 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-02-10T06:37:25.369 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:25.369 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 171 k 2024-02-10T06:37:25.369 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 494 k 2024-02-10T06:37:25.369 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-02-10T06:37:25.416 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-fuse-19.0.0-1361.g290212a1.el9.x86_64.rpm 372 kB/s | 85 kB 00:00 2024-02-10T06:37:25.416 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:25.416 INFO:teuthology.orchestra.run.smithi174.stdout:Total 370 kB/s | 85 kB 00:00 2024-02-10T06:37:25.417 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:25.423 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:25.423 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:25.478 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:25.478 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:25.618 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-nbd-19.0.0-1361.g290212a1.el9.x86_64.rpm 686 kB/s | 171 kB 00:00 2024-02-10T06:37:25.619 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:25.620 INFO:teuthology.orchestra.run.smithi184.stdout:Total 682 kB/s | 171 kB 00:00 2024-02-10T06:37:25.620 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-02-10T06:37:25.623 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:25.626 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-02-10T06:37:25.627 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-02-10T06:37:25.681 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-02-10T06:37:25.682 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-02-10T06:37:25.723 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-02-10T06:37:25.934 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:26.169 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:26.386 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:26.468 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:26.468 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:26.468 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:26.468 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:26.468 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:26.468 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:26.676 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-mirror 2024-02-10T06:37:26.724 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:26.724 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:26.724 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-02-10T06:37:26.724 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:26.724 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:37:26.724 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-02-10T06:37:26.978 DEBUG:teuthology.parallel:result is None 2024-02-10T06:37:27.272 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:55 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:27.431 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:27.431 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:27.431 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 3.1 M 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 13 M 2024-02-10T06:37:27.432 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:28.343 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-mirror-19.0.0-1361.g290212a1.el9.x86_64.rpm 3.4 MB/s | 3.1 MB 00:00 2024-02-10T06:37:28.343 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:28.344 INFO:teuthology.orchestra.run.smithi174.stdout:Total 3.4 MB/s | 3.1 MB 00:00 2024-02-10T06:37:28.344 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:28.353 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:28.353 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:28.426 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:28.426 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:28.853 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:28.910 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:28.934 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:28.935 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T06:37:28.935 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-10T06:37:28.935 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T06:37:28.935 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T06:37:28.935 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:30.075 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:30.076 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:30.076 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:30.076 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:30.076 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:30.076 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:30.294 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-nbd 2024-02-10T06:37:30.889 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:58 ago on Sat 10 Feb 2024 06:35:32 AM UTC. 2024-02-10T06:37:31.050 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 171 k 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 171 k 2024-02-10T06:37:31.051 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 494 k 2024-02-10T06:37:31.052 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-02-10T06:37:31.637 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-nbd-19.0.0-1361.g290212a1.el9.x86_64.rpm 292 kB/s | 171 kB 00:00 2024-02-10T06:37:31.638 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:37:31.638 INFO:teuthology.orchestra.run.smithi174.stdout:Total 291 kB/s | 171 kB 00:00 2024-02-10T06:37:31.638 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-02-10T06:37:31.646 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-02-10T06:37:31.646 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-02-10T06:37:31.702 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-02-10T06:37:31.702 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-02-10T06:37:31.848 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-02-10T06:37:31.978 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:32.464 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:32.792 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T06:37:32.792 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:32.792 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-02-10T06:37:32.792 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T06:37:32.792 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:37:32.792 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-02-10T06:37:33.032 DEBUG:teuthology.parallel:result is None 2024-02-10T06:37:33.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:37:33.208 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-10T06:37:33.229 INFO:teuthology.orchestra.run.smithi174.stdout:19.0.0-1361.g290212a1.el9 2024-02-10T06:37:33.230 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1361.g290212a1.el9 2024-02-10T06:37:33.230 INFO:teuthology.task.install:The correct ceph version 19.0.0-1361.g290212a1 is installed. 2024-02-10T06:37:33.232 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:37:33.390 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-10T06:37:33.414 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-1361.g290212a1.el9 2024-02-10T06:37:33.414 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1361.g290212a1.el9 2024-02-10T06:37:33.415 INFO:teuthology.task.install:The correct ceph version 19.0.0-1361.g290212a1 is installed. 2024-02-10T06:37:33.416 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-10T06:37:33.417 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:37:33.417 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-10T06:37:33.446 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:37:33.447 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-10T06:37:33.486 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-10T06:37:33.486 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:37:33.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-02-10T06:37:33.514 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-10T06:37:33.591 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:37:33.591 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-02-10T06:37:33.620 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-10T06:37:33.691 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-10T06:37:33.691 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:37:33.691 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-10T06:37:33.718 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-10T06:37:33.785 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:37:33.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-10T06:37:33.814 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-10T06:37:33.883 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-10T06:37:33.883 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:37:33.883 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-02-10T06:37:33.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-10T06:37:33.975 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:37:33.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-02-10T06:37:34.002 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-10T06:37:34.070 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-10T06:37:34.171 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'cephadm_mode': 'cephadm-package'} 2024-02-10T06:37:34.172 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:37:34.172 INFO:tasks.cephadm:Cluster fsid is df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:37:34.172 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-10T06:37:34.172 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.174', 'mon.c': '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]', 'mon.b': '172.21.15.184'} 2024-02-10T06:37:34.172 INFO:tasks.cephadm:First mon is mon.a on smithi174 2024-02-10T06:37:34.173 INFO:tasks.cephadm:First mgr is y 2024-02-10T06:37:34.173 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-10T06:37:34.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-02-10T06:37:34.200 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-02-10T06:37:34.239 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 on all hosts... 2024-02-10T06:37:34.239 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T06:37:34.242 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T06:37:35.119 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:37:35.129 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:38:04.223 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-02-10T06:38:04.223 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:38:04.223 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T06:38:04.224 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-02-10T06:38:04.224 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T06:38:04.224 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-02-10T06:38:04.224 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-02-10T06:38:06.573 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-02-10T06:38:06.603 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-02-10T06:38:06.639 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-02-10T06:38:06.671 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-02-10T06:38:06.708 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-02-10T06:38:06.740 INFO:tasks.cephadm:Writing seed config... 2024-02-10T06:38:06.741 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-10T06:38:06.741 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-10T06:38:06.741 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-10T06:38:06.741 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-10T06:38:06.741 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-02-10T06:38:06.742 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-10T06:38:06.743 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:38:06.743 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-10T06:38:06.765 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = df845a14-c7de-11ee-95b8-87774f69a715 mon election default strategy = 1 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-10T06:38:06.765 DEBUG:teuthology.orchestra.run.smithi174:mon.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a.service 2024-02-10T06:38:06.807 DEBUG:teuthology.orchestra.run.smithi174:mgr.y> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y.service 2024-02-10T06:38:06.849 INFO:tasks.cephadm:Bootstrapping... 2024-02-10T06:38:06.850 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -v bootstrap --fsid df845a14-c7de-11ee-95b8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.174 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:38:07.042 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-02-10T06:38:07.043 INFO:teuthology.orchestra.run.smithi174.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', '-v', 'bootstrap', '--fsid', 'df845a14-c7de-11ee-95b8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.174', '--skip-admin-label'] 2024-02-10T06:38:07.068 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T06:38:07.123 INFO:teuthology.orchestra.run.smithi174.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying podman|docker is present... 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying lvm2 is present... 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying time synchronization is in place... 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout inactive 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout enabled 2024-02-10T06:38:07.125 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout active 2024-02-10T06:38:07.126 INFO:teuthology.orchestra.run.smithi174.stdout:Unit chronyd.service is enabled and running 2024-02-10T06:38:07.126 INFO:teuthology.orchestra.run.smithi174.stdout:Repeating the final host check... 2024-02-10T06:38:07.140 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T06:38:07.140 INFO:teuthology.orchestra.run.smithi174.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-10T06:38:07.140 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl is present 2024-02-10T06:38:07.141 INFO:teuthology.orchestra.run.smithi174.stdout:lvcreate is present 2024-02-10T06:38:07.148 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T06:38:07.148 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T06:38:07.152 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T06:38:07.152 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout inactive 2024-02-10T06:38:07.158 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout enabled 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout active 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:Unit chronyd.service is enabled and running 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:Host looks OK 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:Cluster fsid: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:Acquiring lock 140596631343056 on /run/cephadm/df845a14-c7de-11ee-95b8-87774f69a715.lock 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:Lock 140596631343056 acquired on /run/cephadm/df845a14-c7de-11ee-95b8-87774f69a715.lock 2024-02-10T06:38:07.165 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying IP 172.21.15.174 port 3300 ... 2024-02-10T06:38:07.166 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying IP 172.21.15.174 port 6789 ... 2024-02-10T06:38:07.166 INFO:teuthology.orchestra.run.smithi174.stdout:Base mon IP(s) is [172.21.15.174:3300, 172.21.15.174:6789], mon addrv is [v2:172.21.15.174:3300,v1:172.21.15.174:6789] 2024-02-10T06:38:07.168 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.174 metric 100 2024-02-10T06:38:07.168 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.174 metric 100 2024-02-10T06:38:07.171 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-10T06:38:07.171 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-10T06:38:07.171 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1674sec hoplimit 64 pref medium 2024-02-10T06:38:07.173 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-10T06:38:07.173 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-10T06:38:07.173 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T06:38:07.174 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-10T06:38:07.174 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:771b/64 scope link 2024-02-10T06:38:07.174 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T06:38:07.174 INFO:teuthology.orchestra.run.smithi174.stdout:Mon IP `172.21.15.174` is in CIDR network `172.21.0.0/20` 2024-02-10T06:38:07.175 INFO:teuthology.orchestra.run.smithi174.stdout:Mon IP `172.21.15.174` is in CIDR network `172.21.0.0/20` 2024-02-10T06:38:07.175 INFO:teuthology.orchestra.run.smithi174.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-10T06:38:07.175 INFO:teuthology.orchestra.run.smithi174.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-10T06:38:07.176 INFO:teuthology.orchestra.run.smithi174.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stdout c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stderr Getting image source signatures 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stderr Copying blob sha256:5fe0e97a267976e17c7dcb240cf9455472de6876101ae78af4bdf182158b6d8d 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stderr Copying config sha256:c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T06:38:07.700 INFO:teuthology.orchestra.run.smithi174.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-10T06:38:08.513 INFO:teuthology.orchestra.run.smithi174.stdout:ceph: stdout ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T06:38:08.513 INFO:teuthology.orchestra.run.smithi174.stdout:Ceph version: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T06:38:08.513 INFO:teuthology.orchestra.run.smithi174.stdout:Extracting ceph user uid/gid from container image... 2024-02-10T06:38:09.361 INFO:teuthology.orchestra.run.smithi174.stdout:stat: stdout 167 167 2024-02-10T06:38:09.362 INFO:teuthology.orchestra.run.smithi174.stdout:Creating initial keys... 2024-02-10T06:38:10.318 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph-authtool: stdout AQDRGcdl6Z0TLBAATe0i4hrxO5Lsp5XJQgUdgA== 2024-02-10T06:38:11.147 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph-authtool: stdout AQDSGcdlzAToIxAA0larRVGvKYENGKtHjs86Mg== 2024-02-10T06:38:12.002 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph-authtool: stdout AQDTGcdlzzFJGhAAfm6/5XBy2007cYjw4apCkw== 2024-02-10T06:38:12.002 INFO:teuthology.orchestra.run.smithi174.stdout:Creating initial monmap... 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool for a [v2:172.21.15.174:3300,v1:172.21.15.174:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:setting min_mon_release = quincy 2024-02-10T06:38:17.960 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/monmaptool: set fsid to df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:17.961 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T06:38:17.961 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:38:17.961 INFO:teuthology.orchestra.run.smithi174.stdout:Creating mon... 2024-02-10T06:38:24.058 INFO:teuthology.orchestra.run.smithi174.stdout:create mon.a on 2024-02-10T06:38:24.259 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-10T06:38:24.420 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-10T06:38:24.613 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-df845a14-c7de-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-df845a14-c7de-11ee-95b8-87774f69a715.target. 2024-02-10T06:38:24.613 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-df845a14-c7de-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-df845a14-c7de-11ee-95b8-87774f69a715.target. 2024-02-10T06:38:24.854 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a 2024-02-10T06:38:24.854 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Failed to reset failed state of unit ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a.service: Unit ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a.service not loaded. 2024-02-10T06:38:25.020 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-df845a14-c7de-11ee-95b8-87774f69a715.target.wants/ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-df845a14-c7de-11ee-95b8-87774f69a715@.service. 2024-02-10T06:38:25.457 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 podman[29364]: 2024-02-10 06:38:25.349219926 +0000 UTC m=+0.191066920 container create 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-10T06:38:25.698 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:38:25.699 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout disabled 2024-02-10T06:38:25.705 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:38:25.705 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout inactive 2024-02-10T06:38:25.705 INFO:teuthology.orchestra.run.smithi174.stdout:firewalld.service is not enabled 2024-02-10T06:38:25.705 INFO:teuthology.orchestra.run.smithi174.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T06:38:25.706 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for mon to start... 2024-02-10T06:38:25.706 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for mon... 2024-02-10T06:38:25.830 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 podman[29364]: 2024-02-10 06:38:25.576387227 +0000 UTC m=+0.418234222 container init 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:38:25.831 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 podman[29364]: 2024-02-10 06:38:25.583525965 +0000 UTC m=+0.425372960 container start 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:38:25.831 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 ceph-mon[29402]: mkfs df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:25.831 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 ceph-mon[29402]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T06:38:25.831 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 bash[29364]: 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b 2024-02-10T06:38:25.831 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:25 smithi174 systemd[1]: Started Ceph mon.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:38:26.791 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout cluster: 2024-02-10T06:38:26.791 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout id: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:26.791 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-10T06:38:26.791 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout services: 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.576952s) 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout data: 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout pgs: 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.148+0000 7efd432a2700 1 Processor -- start 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.148+0000 7efd432a2700 1 -- start start 2024-02-10T06:38:26.792 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.148+0000 7efd432a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c105180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.148+0000 7efd432a2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c105750 con 0x7efd3c104d60 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd4103e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c105180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd4103e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c105180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47036/0 (socket says 172.21.15.174:47036) 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd4103e700 1 -- 172.21.15.174:0/109003187 learned_addr learned my addr 172.21.15.174:0/109003187 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd4103e700 1 -- 172.21.15.174:0/109003187 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c105f60 con 0x7efd3c104d60 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd4103e700 1 --2- 172.21.15.174:0/109003187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c105180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e80 tx=0x7efd3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c387b7a87839a943 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd3bfff700 1 -- 172.21.15.174:0/109003187 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd3400e070 con 0x7efd3c104d60 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd3bfff700 1 -- 172.21.15.174:0/109003187 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34003a20 con 0x7efd3c104d60 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.149+0000 7efd3bfff700 1 -- 172.21.15.174:0/109003187 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd34003ce0 con 0x7efd3c104d60 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 -- 172.21.15.174:0/109003187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 msgr2=0x7efd3c105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 --2- 172.21.15.174:0/109003187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c105180 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e80 tx=0x7efd3400b330 comp rx=0 tx=0).stop 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 -- 172.21.15.174:0/109003187 shutdown_connections 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 --2- 172.21.15.174:0/109003187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c105180 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 -- 172.21.15.174:0/109003187 >> 172.21.15.174:0/109003187 conn(0x7efd3c100300 msgr2=0x7efd3c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 -- 172.21.15.174:0/109003187 shutdown_connections 2024-02-10T06:38:26.793 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.150+0000 7efd432a2700 1 -- 172.21.15.174:0/109003187 wait complete. 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd432a2700 1 Processor -- start 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd432a2700 1 -- start start 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd432a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c0792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd432a2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c079820 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd4103e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c0792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd4103e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c0792e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47048/0 (socket says 172.21.15.174:47048) 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd4103e700 1 -- 172.21.15.174:0/1026100202 learned_addr learned my addr 172.21.15.174:0/1026100202 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd4103e700 1 -- 172.21.15.174:0/1026100202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009b30 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd4103e700 1 --2- 172.21.15.174:0/1026100202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c0792e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efd34004000 tx=0x7efd34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd34004370 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34004510 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd340047f0 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c19c020 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.151+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c19c4c0 con 0x7efd3c104d60 2024-02-10T06:38:26.794 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.152+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7efd340127c0 con 0x7efd3c104d60 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.152+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd3401dbf0 con 0x7efd3c104d60 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.152+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c191f20 con 0x7efd3c104d60 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.155+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efd3402b060 con 0x7efd3c104d60 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.224+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7efd3c106500 con 0x7efd3c104d60 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.224+0000 7efd3a7fc700 1 -- 172.21.15.174:0/1026100202 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7efd34021030 con 0x7efd3c104d60 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 msgr2=0x7efd3c0792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 --2- 172.21.15.174:0/1026100202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c0792e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efd34004000 tx=0x7efd34004ad0 comp rx=0 tx=0).stop 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 shutdown_connections 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 --2- 172.21.15.174:0/1026100202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd3c104d60 0x7efd3c0792e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 >> 172.21.15.174:0/1026100202 conn(0x7efd3c100300 msgr2=0x7efd3c18f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 shutdown_connections 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:26.225+0000 7efd432a2700 1 -- 172.21.15.174:0/1026100202 wait complete. 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:mon is available 2024-02-10T06:38:26.795 INFO:teuthology.orchestra.run.smithi174.stdout:Assimilating anything we can from ceph.conf... 2024-02-10T06:38:26.913 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:26 smithi174 ceph-mon[29402]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T06:38:26.913 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:26 smithi174 ceph-mon[29402]: monmap e1: 1 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:38:26.913 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:26 smithi174 ceph-mon[29402]: fsmap 2024-02-10T06:38:26.913 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:26 smithi174 ceph-mon[29402]: osdmap e1: 0 total, 0 up, 0 in 2024-02-10T06:38:26.913 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:26 smithi174 ceph-mon[29402]: mgrmap e1: no daemons active 2024-02-10T06:38:26.913 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:26 smithi174 ceph-mon[29402]: from='client.? 172.21.15.174:0/1026100202' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-10T06:38:27.981 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:27 smithi174 ceph-mon[29402]: from='client.? 172.21.15.174:0/2923568124' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T06:38:27.981 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:27 smithi174 ceph-mon[29402]: from='client.? 172.21.15.174:0/2923568124' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout fsid = df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.174:3300,v1:172.21.15.174:6789] 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T06:38:27.989 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.295+0000 7f76a763b700 1 Processor -- start 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.295+0000 7f76a763b700 1 -- start start 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.295+0000 7f76a763b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a0105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.295+0000 7f76a763b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01056f0 con 0x7f76a0104d00 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f76a53d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a0105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f76a53d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a0105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47062/0 (socket says 172.21.15.174:47062) 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f76a53d7700 1 -- 172.21.15.174:0/4004781513 learned_addr learned my addr 172.21.15.174:0/4004781513 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f76a53d7700 1 -- 172.21.15.174:0/4004781513 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0105f00 con 0x7f76a0104d00 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f76a53d7700 1 --2- 172.21.15.174:0/4004781513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a0105120 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f769c0098a0 tx=0x7f769c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a679928355201ad1 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f7697fff700 1 -- 172.21.15.174:0/4004781513 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f769c004160 con 0x7f76a0104d00 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f7697fff700 1 -- 172.21.15.174:0/4004781513 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c0042c0 con 0x7f76a0104d00 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.296+0000 7f7697fff700 1 -- 172.21.15.174:0/4004781513 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f769c0045a0 con 0x7f76a0104d00 2024-02-10T06:38:27.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 -- 172.21.15.174:0/4004781513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 msgr2=0x7f76a0105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 --2- 172.21.15.174:0/4004781513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a0105120 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f769c0098a0 tx=0x7f769c009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 -- 172.21.15.174:0/4004781513 shutdown_connections 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 --2- 172.21.15.174:0/4004781513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a0105120 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 -- 172.21.15.174:0/4004781513 >> 172.21.15.174:0/4004781513 conn(0x7f76a01002c0 msgr2=0x7f76a01026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 -- 172.21.15.174:0/4004781513 shutdown_connections 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.297+0000 7f76a763b700 1 -- 172.21.15.174:0/4004781513 wait complete. 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a763b700 1 Processor -- start 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a763b700 1 -- start start 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a763b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a763b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01986f0 con 0x7f76a0104d00 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a53d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a53d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47078/0 (socket says 172.21.15.174:47078) 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a53d7700 1 -- 172.21.15.174:0/2923568124 learned_addr learned my addr 172.21.15.174:0/2923568124 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a53d7700 1 -- 172.21.15.174:0/2923568124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c009550 con 0x7f76a0104d00 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.298+0000 7f76a53d7700 1 --2- 172.21.15.174:0/2923568124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a01981b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f769c004000 tx=0x7f769c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f769c011630 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c011790 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01988f0 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f769c011a30 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a0198d90 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f769c021070 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f769c01b870 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.299+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a0191e80 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.302+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f769c01bad0 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.368+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f76a0106270 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.370+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+340 (secure 0 0 0) 0x7f769c01bd20 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76967fc700 1 -- 172.21.15.174:0/2923568124 <== mon.0 v2:172.21.15.174:3300/0 8 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f769c01a460 con 0x7f76a0104d00 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 msgr2=0x7f76a01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 --2- 172.21.15.174:0/2923568124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a01981b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f769c004000 tx=0x7f769c004750 comp rx=0 tx=0).stop 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 shutdown_connections 2024-02-10T06:38:27.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 --2- 172.21.15.174:0/2923568124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0104d00 0x7f76a01981b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:27.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 >> 172.21.15.174:0/2923568124 conn(0x7f76a01002c0 msgr2=0x7f76a018efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:27.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 shutdown_connections 2024-02-10T06:38:27.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:27.371+0000 7f76a763b700 1 -- 172.21.15.174:0/2923568124 wait complete. 2024-02-10T06:38:27.993 INFO:teuthology.orchestra.run.smithi174.stdout:Generating new minimal ceph.conf... 2024-02-10T06:38:28.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:28 smithi174 ceph-mon[29402]: from='client.? 172.21.15.174:0/3912709172' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.462+0000 7f5620743700 1 Processor -- start 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.462+0000 7f5620743700 1 -- start start 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f5620743700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f5618104af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f5620743700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180796e0 con 0x7f56181046d0 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561e4df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f5618104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561e4df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f5618104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47086/0 (socket says 172.21.15.174:47086) 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561e4df700 1 -- 172.21.15.174:0/2511933064 learned_addr learned my addr 172.21.15.174:0/2511933064 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:31.085 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561e4df700 1 -- 172.21.15.174:0/2511933064 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618105030 con 0x7f56181046d0 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561e4df700 1 --2- 172.21.15.174:0/2511933064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f5618104af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f56140098a0 tx=0x7f5614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5035dcb6bfbfc5f server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561d4dd700 1 -- 172.21.15.174:0/2511933064 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5614004160 con 0x7f56181046d0 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561d4dd700 1 -- 172.21.15.174:0/2511933064 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f56140042c0 con 0x7f56181046d0 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.463+0000 7f561d4dd700 1 -- 172.21.15.174:0/2511933064 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56140119b0 con 0x7f56181046d0 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 -- 172.21.15.174:0/2511933064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 msgr2=0x7f5618104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 --2- 172.21.15.174:0/2511933064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f5618104af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f56140098a0 tx=0x7f5614009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 -- 172.21.15.174:0/2511933064 shutdown_connections 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 --2- 172.21.15.174:0/2511933064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f5618104af0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 -- 172.21.15.174:0/2511933064 >> 172.21.15.174:0/2511933064 conn(0x7f5618100270 msgr2=0x7f56181026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 -- 172.21.15.174:0/2511933064 shutdown_connections 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 -- 172.21.15.174:0/2511933064 wait complete. 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.464+0000 7f5620743700 1 Processor -- start 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f5620743700 1 -- start start 2024-02-10T06:38:31.086 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f5620743700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f561819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f5620743700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819c1a0 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f561e4df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f561819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f561e4df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f561819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47090/0 (socket says 172.21.15.174:47090) 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f561e4df700 1 -- 172.21.15.174:0/3912709172 learned_addr learned my addr 172.21.15.174:0/3912709172 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f561e4df700 1 -- 172.21.15.174:0/3912709172 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614009550 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f561e4df700 1 --2- 172.21.15.174:0/3912709172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f561819bc60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5614009130 tx=0x7f5614011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5614003aa0 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f561401a8c0 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561819c3a0 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5614019790 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.465+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561819c840 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.466+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5614025460 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.466+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f561400bb20 con 0x7f56181046d0 2024-02-10T06:38:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.466+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618195950 con 0x7f56181046d0 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.469+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f561402a060 con 0x7f56181046d0 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.536+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5618105ee0 con 0x7f56181046d0 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.537+0000 7f560f7fe700 1 -- 172.21.15.174:0/3912709172 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f561401aa30 con 0x7f56181046d0 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.537+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 msgr2=0x7f561819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.537+0000 7f5620743700 1 --2- 172.21.15.174:0/3912709172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f561819bc60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5614009130 tx=0x7f5614011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.538+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 shutdown_connections 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.538+0000 7f5620743700 1 --2- 172.21.15.174:0/3912709172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56181046d0 0x7f561819bc60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.538+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 >> 172.21.15.174:0/3912709172 conn(0x7f5618100270 msgr2=0x7f5618192a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.538+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 shutdown_connections 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:28.538+0000 7f5620743700 1 -- 172.21.15.174:0/3912709172 wait complete. 2024-02-10T06:38:31.088 INFO:teuthology.orchestra.run.smithi174.stdout:Restarting the monitor... 2024-02-10T06:38:31.359 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:31 smithi174 systemd[1]: Stopping Ceph mon.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:38:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a[29379]: 2024-02-10T06:38:31.357+0000 7fa8d0aab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T06:38:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a[29379]: 2024-02-10T06:38:31.357+0000 7fa8d0aab700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-10T06:38:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:31 smithi174 podman[29722]: 2024-02-10 06:38:31.603710683 +0000 UTC m=+0.352698381 container died 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-10T06:38:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:31 smithi174 podman[29722]: 2024-02-10 06:38:31.93002933 +0000 UTC m=+0.679017037 container cleanup 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-10T06:38:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:31 smithi174 bash[29722]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a 2024-02-10T06:38:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 podman[29741]: 2024-02-10 06:38:32.089146319 +0000 UTC m=+0.489238261 container remove 01231a1594da0962a6ef3489331c3913cf37cfa1e53a3af15c45a459aef88c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, io.buildah.version=1.33.2, ceph=True, RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-10T06:38:32.626 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-10T06:38:32.626 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 systemd[1]: Stopped Ceph mon.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:38:32.626 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 systemd[1]: Starting Ceph mon.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:38:32.626 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 podman[29841]: 2024-02-10 06:38:32.514925561 +0000 UTC m=+0.113533944 container create 5b25e513001d1364c67f23b69bb4f8411ba20be74c137dba63183dd792e31412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, RELEASE=main-290212a) 2024-02-10T06:38:32.626 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 podman[29841]: 2024-02-10 06:38:32.419287152 +0000 UTC m=+0.017895534 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:38:32.788 INFO:teuthology.orchestra.run.smithi174.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-10T06:38:32.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 podman[29841]: 2024-02-10 06:38:32.700472763 +0000 UTC m=+0.299081158 container init 5b25e513001d1364c67f23b69bb4f8411ba20be74c137dba63183dd792e31412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 podman[29841]: 2024-02-10 06:38:32.705918325 +0000 UTC m=+0.304526708 container start 5b25e513001d1364c67f23b69bb4f8411ba20be74c137dba63183dd792e31412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: pidfile_write: ignore empty --pid-file 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: load: jerasure load: lrc 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: RocksDB version: 7.9.2 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Git sha 0 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: DB SUMMARY 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: DB Session ID: 5QGG8ZD407GFM8CM8ZC4 2024-02-10T06:38:32.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: CURRENT file: CURRENT 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88230 ; 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.error_if_exists: 0 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.create_if_missing: 0 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.paranoid_checks: 1 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.env: 0x55b0f4228ae0 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.info_log: 0x55b0f5fd98c0 2024-02-10T06:38:32.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.statistics: (nil) 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.use_fsync: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_log_file_size: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.allow_fallocate: 1 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.use_direct_reads: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T06:38:32.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.db_log_dir: 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.wal_dir: 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.write_buffer_manager: 0x55b0f5fad400 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T06:38:32.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.unordered_write: 0 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.row_cache: None 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.wal_filter: None 2024-02-10T06:38:32.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T06:38:32.995 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.two_write_queues: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.wal_compression: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.atomic_flush: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.log_readahead_size: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T06:38:32.996 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_background_jobs: 2 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_background_compactions: -1 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_subcompactions: 1 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_open_files: -1 2024-02-10T06:38:32.997 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_background_flushes: -1 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Compression algorithms supported: 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kZSTD supported: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kXpressCompression supported: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kZlibCompression supported: 1 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kSnappyCompression supported: 1 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kLZ4Compression supported: 1 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: kBZip2Compression supported: 0 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T06:38:32.998 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.merge_operator: 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_filter: None 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b0f5fd99c0) 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-02-10T06:38:32.999 INFO:journalctl@ceph.mon.a.smithi174.stdout: index_type: 0 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: data_block_index_type: 0 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: index_shortening: 1 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: checksum: 4 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: no_block_cache: 0 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_cache: 0x55b0f5fdd090 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_cache_options: 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: capacity : 536870912 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: num_shard_bits : 4 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: strict_capacity_limit : 0 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_cache_compressed: (nil) 2024-02-10T06:38:33.000 INFO:journalctl@ceph.mon.a.smithi174.stdout: persistent_cache: (nil) 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_size: 4096 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_size_deviation: 10 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_restart_interval: 16 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: index_block_restart_interval: 1 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: metadata_block_size: 4096 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: partition_filters: 0 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: use_delta_encoding: 1 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: filter_policy: bloomfilter 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: whole_key_filtering: 1 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: verify_compression: 0 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: format_version: 5 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: enable_index_compression: 1 2024-02-10T06:38:33.001 INFO:journalctl@ceph.mon.a.smithi174.stdout: block_align: 0 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout: max_auto_readahead_size: 262144 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout: prepopulate_block_cache: 0 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout: initial_auto_readahead_size: 8192 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression: NoCompression 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.num_levels: 7 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T06:38:33.002 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:38:33.003 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T06:38:33.004 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T06:38:33.005 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.table_properties_collectors: 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.inplace_update_support: 0 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T06:38:33.006 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.bloom_locality: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.max_successive_merges: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.ttl: 2592000 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.enable_blob_files: false 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.min_blob_size: 0 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T06:38:33.007 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1b34707d-8366-4afa-831d-11fa553592c7 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547112734229, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547112735087, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 84234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 275, "table_properties": {"data_size": 82413, "index_size": 231, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10830, "raw_average_key_size": 47, "raw_value_size": 76377, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 226, "num_filter_entries": 226, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707547112, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1b34707d-8366-4afa-831d-11fa553592c7", "db_session_id": "5QGG8ZD407GFM8CM8ZC4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547112735168, "job": 1, "event": "recovery_finished"} 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T06:38:33.008 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b0f60ce000 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: DB pointer 0x55b0f60b8000 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: ** DB Stats ** 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: ** Compaction Stats [default] ** 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:38:33.009 INFO:journalctl@ceph.mon.a.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: L0 2/0 84.09 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: Sum 2/0 84.09 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: ** Compaction Stats [default] ** 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 149.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T06:38:33.010 INFO:journalctl@ceph.mon.a.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: Cumulative compaction: 0.00 GB write, 29.10 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: Interval compaction: 0.00 GB write, 29.10 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: Block cache BinnedLRUCache@0x55b0f5fdd090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: starting mon.a rank 0 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???) e1 preinit fsid df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).mds e1 new map 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).mds e1 print_map 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: e1 2024-02-10T06:38:33.011 INFO:journalctl@ceph.mon.a.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout: legacy client fscid: -1 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout: 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout: No filesystems configured 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 bash[29841]: 5b25e513001d1364c67f23b69bb4f8411ba20be74c137dba63183dd792e31412 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 systemd[1]: Started Ceph mon.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: monmap e1: 1 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: fsmap 2024-02-10T06:38:33.012 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: osdmap e1: 0 total, 0 up, 0 in 2024-02-10T06:38:33.013 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:32 smithi174 ceph-mon[29878]: mgrmap e1: no daemons active 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.248+0000 7f93508c0700 1 Processor -- start 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.248+0000 7f93508c0700 1 -- start start 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f93508c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f93508c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c106d00 con 0x7f934c106310 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f934a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f934a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47092/0 (socket says 172.21.15.174:47092) 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f934a59c700 1 -- 172.21.15.174:0/2475454358 learned_addr learned my addr 172.21.15.174:0/2475454358 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f934a59c700 1 -- 172.21.15.174:0/2475454358 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c106e40 con 0x7f934c106310 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.249+0000 7f934a59c700 1 --2- 172.21.15.174:0/2475454358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c106730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f93400098a0 tx=0x7f9340009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa16e73c031b8d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f934959a700 1 -- 172.21.15.174:0/2475454358 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9340004160 con 0x7f934c106310 2024-02-10T06:38:33.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f934959a700 1 -- 172.21.15.174:0/2475454358 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f93400042c0 con 0x7f934c106310 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f934959a700 1 -- 172.21.15.174:0/2475454358 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93400119b0 con 0x7f934c106310 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 -- 172.21.15.174:0/2475454358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 msgr2=0x7f934c106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 --2- 172.21.15.174:0/2475454358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c106730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f93400098a0 tx=0x7f9340009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 -- 172.21.15.174:0/2475454358 shutdown_connections 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 --2- 172.21.15.174:0/2475454358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c106730 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 -- 172.21.15.174:0/2475454358 >> 172.21.15.174:0/2475454358 conn(0x7f934c0ffa40 msgr2=0x7f934c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 -- 172.21.15.174:0/2475454358 shutdown_connections 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.250+0000 7f93508c0700 1 -- 172.21.15.174:0/2475454358 wait complete. 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f93508c0700 1 Processor -- start 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f93508c0700 1 -- start start 2024-02-10T06:38:33.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f93508c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f93508c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c19c1f0 con 0x7f934c106310 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f934a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f934a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47098/0 (socket says 172.21.15.174:47098) 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.251+0000 7f934a59c700 1 -- 172.21.15.174:0/977043902 learned_addr learned my addr 172.21.15.174:0/977043902 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f934a59c700 1 -- 172.21.15.174:0/977043902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009550 con 0x7f934c106310 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f934a59c700 1 --2- 172.21.15.174:0/977043902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c19bcb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9340006af0 tx=0x7f9340011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:33.951 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93400039e0 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f9340003b40 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9340019720 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c19c3f0 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c19c890 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9340019ce0 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.252+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f934002c6a0 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.253+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c195a00 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.255+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9340042dd0 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.321+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f934c104be0 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.323+0000 7f933b7fe700 1 -- 172.21.15.174:0/977043902 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9340031020 con 0x7f934c106310 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 msgr2=0x7f934c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 --2- 172.21.15.174:0/977043902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c19bcb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9340006af0 tx=0x7f9340011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:33.952 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 shutdown_connections 2024-02-10T06:38:33.953 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 --2- 172.21.15.174:0/977043902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f934c106310 0x7f934c19bcb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:33.953 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 >> 172.21.15.174:0/977043902 conn(0x7f934c0ffa40 msgr2=0x7f934c192b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:33.953 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 shutdown_connections 2024-02-10T06:38:33.953 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:33.324+0000 7f93508c0700 1 -- 172.21.15.174:0/977043902 wait complete. 2024-02-10T06:38:33.953 INFO:teuthology.orchestra.run.smithi174.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-10T06:38:33.972 INFO:teuthology.orchestra.run.smithi174.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:38:33.972 INFO:teuthology.orchestra.run.smithi174.stdout:Creating mgr... 2024-02-10T06:38:33.973 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-10T06:38:33.973 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-10T06:38:34.221 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y 2024-02-10T06:38:34.221 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Failed to reset failed state of unit ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y.service: Unit ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y.service not loaded. 2024-02-10T06:38:34.363 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-df845a14-c7de-11ee-95b8-87774f69a715.target.wants/ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-df845a14-c7de-11ee-95b8-87774f69a715@.service. 2024-02-10T06:38:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/977043902' entity='client.admin' 2024-02-10T06:38:34.489 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 systemd[1]: Starting Ceph mgr.y for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:38:34.910 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 podman[30115]: 2024-02-10 06:38:34.494236384 +0000 UTC m=+0.017667386 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:38:34.910 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 podman[30115]: 2024-02-10 06:38:34.591480438 +0000 UTC m=+0.114911433 container create 28c10ca2cfc84b3391e8be5298b088d775e21b24d3dbd1361e83898b123fd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.schema-version=1.0) 2024-02-10T06:38:34.910 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 podman[30115]: 2024-02-10 06:38:34.810143685 +0000 UTC m=+0.333574677 container init 28c10ca2cfc84b3391e8be5298b088d775e21b24d3dbd1361e83898b123fd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-10T06:38:34.910 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 podman[30115]: 2024-02-10 06:38:34.815697359 +0000 UTC m=+0.339128351 container start 28c10ca2cfc84b3391e8be5298b088d775e21b24d3dbd1361e83898b123fd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:38:34.910 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 bash[30115]: 28c10ca2cfc84b3391e8be5298b088d775e21b24d3dbd1361e83898b123fd074 2024-02-10T06:38:34.923 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:38:34.923 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout disabled 2024-02-10T06:38:34.930 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:38:34.931 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout inactive 2024-02-10T06:38:34.931 INFO:teuthology.orchestra.run.smithi174.stdout:firewalld.service is not enabled 2024-02-10T06:38:34.931 INFO:teuthology.orchestra.run.smithi174.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T06:38:34.939 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:38:34.939 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout disabled 2024-02-10T06:38:34.946 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:38:34.946 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout inactive 2024-02-10T06:38:34.946 INFO:teuthology.orchestra.run.smithi174.stdout:firewalld.service is not enabled 2024-02-10T06:38:34.946 INFO:teuthology.orchestra.run.smithi174.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-10T06:38:34.946 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for mgr to start... 2024-02-10T06:38:34.946 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for mgr... 2024-02-10T06:38:35.203 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:34 smithi174 systemd[1]: Started Ceph mgr.y for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:38:35.459 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:35 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:35.370+0000 7f73f4247200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:38:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2231944001' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:38:35.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:35 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:35.468+0000 7f73f4247200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:38:36.383 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.118+0000 7f73f4247200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:38:36.383 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.212+0000 7f73f4247200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:38:36.383 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.354+0000 7f73f4247200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:38:36.386 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:36.388 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:38:36.388 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsid": "df845a14-c7de-11ee-95b8-87774f69a715", 2024-02-10T06:38:36.388 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:38:36.388 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:38:36.388 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:38:36.389 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.390 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:38:36.390 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:36.390 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:38:36.390 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:38:36.390 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:38:36.397 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T06:38:36.398 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:38:25.650575+0000", 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:38:36.399 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.458+0000 7fc1f386a700 1 Processor -- start 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.458+0000 7fc1f386a700 1 -- start start 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.458+0000 7fc1f386a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec104ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.458+0000 7fc1f386a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec078e50 con 0x7fc1ec1046c0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1f1606700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec104ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1f1606700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec104ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38846/0 (socket says 172.21.15.174:38846) 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1f1606700 1 -- 172.21.15.174:0/1539881975 learned_addr learned my addr 172.21.15.174:0/1539881975 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1f1606700 1 -- 172.21.15.174:0/1539881975 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec105020 con 0x7fc1ec1046c0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1f1606700 1 --2- 172.21.15.174:0/1539881975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec104ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80098a0 tx=0x7fc1e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e007c5683c46ffff server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1e3fff700 1 -- 172.21.15.174:0/1539881975 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e8004160 con 0x7fc1ec1046c0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1e3fff700 1 -- 172.21.15.174:0/1539881975 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fc1e80042c0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.459+0000 7fc1e3fff700 1 -- 172.21.15.174:0/1539881975 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e80119b0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 -- 172.21.15.174:0/1539881975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 msgr2=0x7fc1ec104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 --2- 172.21.15.174:0/1539881975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec104ae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80098a0 tx=0x7fc1e8009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:36.400 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 -- 172.21.15.174:0/1539881975 shutdown_connections 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 --2- 172.21.15.174:0/1539881975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec104ae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 -- 172.21.15.174:0/1539881975 >> 172.21.15.174:0/1539881975 conn(0x7fc1ec1002c0 msgr2=0x7fc1ec1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 -- 172.21.15.174:0/1539881975 shutdown_connections 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.460+0000 7fc1f386a700 1 -- 172.21.15.174:0/1539881975 wait complete. 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f386a700 1 Processor -- start 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f386a700 1 -- start start 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f386a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f386a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec19c1c0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f1606700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f1606700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38854/0 (socket says 172.21.15.174:38854) 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f1606700 1 -- 172.21.15.174:0/2231944001 learned_addr learned my addr 172.21.15.174:0/2231944001 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f1606700 1 -- 172.21.15.174:0/2231944001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8009550 con 0x7fc1ec1046c0 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f1606700 1 --2- 172.21.15.174:0/2231944001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec19bc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009130 tx=0x7fc1e8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e8003af0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.401 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fc1e801a8c0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.461+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ec19c3c0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.462+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e8019710 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.462+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec19c860 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.462+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc1e8025460 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.462+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc1e800bb10 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.462+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ec195940 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.465+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc1e802a060 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.538+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc1ec105fc0 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.539+0000 7fc1e27fc700 1 -- 172.21.15.174:0/2231944001 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fc1e801aa30 con 0x7fc1ec1046c0 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 msgr2=0x7fc1ec19bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 --2- 172.21.15.174:0/2231944001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec19bc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009130 tx=0x7fc1e8011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 shutdown_connections 2024-02-10T06:38:36.402 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 --2- 172.21.15.174:0/2231944001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1ec1046c0 0x7fc1ec19bc80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:36.403 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 >> 172.21.15.174:0/2231944001 conn(0x7fc1ec1002c0 msgr2=0x7fc1ec192a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:36.403 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 shutdown_connections 2024-02-10T06:38:36.403 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:35.540+0000 7fc1f386a700 1 -- 172.21.15.174:0/2231944001 wait complete. 2024-02-10T06:38:36.403 INFO:teuthology.orchestra.run.smithi174.stdout:mgr not available, waiting (1/15)... 2024-02-10T06:38:36.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.450+0000 7f73f4247200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:38:36.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.533+0000 7f73f4247200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:38:37.082 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.795+0000 7f73f4247200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:38:37.082 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.902+0000 7f73f4247200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:38:37.082 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:36 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:36.987+0000 7f73f4247200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:38:37.082 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:37.081+0000 7f73f4247200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:38:37.489 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:37.267+0000 7f73f4247200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:38:38.489 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:38 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:38.235+0000 7f73f4247200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:38:38.489 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:38 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:38.365+0000 7f73f4247200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:38:38.808 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:38 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:38.713+0000 7f73f4247200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:38:38.808 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:38 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:38.807+0000 7f73f4247200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:38:39.203 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3636841830' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:38:39.204 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:39.021+0000 7f73f4247200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:38:39.204 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:39.203+0000 7f73f4247200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:38:39.644 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsid": "df845a14-c7de-11ee-95b8-87774f69a715", 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:38:39.645 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:38:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:38:39.654 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:38:39.655 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:38:25.650575+0000", 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb8744f3700 1 Processor -- start 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb8744f3700 1 -- start start 2024-02-10T06:38:39.656 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb8744f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c105180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb8744f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c105750 con 0x7fb86c104d60 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb87228f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c105180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb87228f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c105180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38858/0 (socket says 172.21.15.174:38858) 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.847+0000 7fb87228f700 1 -- 172.21.15.174:0/2286059398 learned_addr learned my addr 172.21.15.174:0/2286059398 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.848+0000 7fb87228f700 1 -- 172.21.15.174:0/2286059398 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c105f60 con 0x7fb86c104d60 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.848+0000 7fb87228f700 1 --2- 172.21.15.174:0/2286059398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c105180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098a0 tx=0x7fb868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28950e28018a102 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.848+0000 7fb87128d700 1 -- 172.21.15.174:0/2286059398 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb868004160 con 0x7fb86c104d60 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.848+0000 7fb87128d700 1 -- 172.21.15.174:0/2286059398 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb8680042c0 con 0x7fb86c104d60 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.848+0000 7fb87128d700 1 -- 172.21.15.174:0/2286059398 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8680119b0 con 0x7fb86c104d60 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 -- 172.21.15.174:0/2286059398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 msgr2=0x7fb86c105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 --2- 172.21.15.174:0/2286059398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c105180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098a0 tx=0x7fb868009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 -- 172.21.15.174:0/2286059398 shutdown_connections 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 --2- 172.21.15.174:0/2286059398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c105180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 -- 172.21.15.174:0/2286059398 >> 172.21.15.174:0/2286059398 conn(0x7fb86c100300 msgr2=0x7fb86c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 -- 172.21.15.174:0/2286059398 shutdown_connections 2024-02-10T06:38:39.657 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 -- 172.21.15.174:0/2286059398 wait complete. 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 Processor -- start 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.849+0000 7fb8744f3700 1 -- start start 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb8744f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb8744f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c198f60 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb87228f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb87228f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38864/0 (socket says 172.21.15.174:38864) 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb87228f700 1 -- 172.21.15.174:0/3636841830 learned_addr learned my addr 172.21.15.174:0/3636841830 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb87228f700 1 -- 172.21.15.174:0/3636841830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb868009550 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb87228f700 1 --2- 172.21.15.174:0/3636841830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c198a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8680046c0 tx=0x7fb868011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8680038b0 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb868003a10 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb868019610 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.850+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c199160 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.851+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c199600 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.851+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb868019880 con 0x7fb86c104d60 2024-02-10T06:38:39.658 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.851+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb868024730 con 0x7fb86c104d60 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.852+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c063350 con 0x7fb86c104d60 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.854+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb868003b80 con 0x7fb86c104d60 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.926+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb86c109140 con 0x7fb86c104d60 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.926+0000 7fb8637fe700 1 -- 172.21.15.174:0/3636841830 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb86802dde0 con 0x7fb86c104d60 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 msgr2=0x7fb86c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 --2- 172.21.15.174:0/3636841830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c198a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8680046c0 tx=0x7fb868011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 shutdown_connections 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 --2- 172.21.15.174:0/3636841830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c104d60 0x7fb86c198a20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 >> 172.21.15.174:0/3636841830 conn(0x7fb86c100300 msgr2=0x7fb86c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 shutdown_connections 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:38.927+0000 7fb8744f3700 1 -- 172.21.15.174:0/3636841830 wait complete. 2024-02-10T06:38:39.659 INFO:teuthology.orchestra.run.smithi174.stdout:mgr not available, waiting (2/15)... 2024-02-10T06:38:39.754 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:39.489+0000 7f73f4247200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:38:39.754 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:39.753+0000 7f73f4247200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:38:40.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:39.986+0000 7f73f4247200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:38:41.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:40 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:40.779+0000 7f73f4247200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:38:41.661 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:41.349+0000 7f73f4247200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:38:41.662 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:41.447+0000 7f73f4247200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:38:41.662 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:41.660+0000 7f73f4247200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:38:41.916 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: Activating manager daemon y 2024-02-10T06:38:41.916 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: mgrmap e2: y(active, starting, since 0.00398437s) 2024-02-10T06:38:41.916 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:38:41.916 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: Manager daemon y is now available 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' 2024-02-10T06:38:41.917 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:41 smithi174 ceph-mon[29878]: from='mgr.14100 172.21.15.174:0/3928632109' entity='mgr.y' 2024-02-10T06:38:42.771 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/95643269' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:38:42.771 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:42 smithi174 ceph-mon[29878]: mgrmap e3: y(active, since 1.0064s) 2024-02-10T06:38:42.775 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:42.775 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:38:42.775 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsid": "df845a14-c7de-11ee-95b8-87774f69a715", 2024-02-10T06:38:42.775 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:38:42.776 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:38:42.777 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.777 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:38:42.777 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:42.777 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:38:42.777 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:38:42.777 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:38:42.784 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:38:42.785 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:38:25.650575+0000", 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1aa5ea6700 1 Processor -- start 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1aa5ea6700 1 -- start start 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1aa5ea6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1aa5ea6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0104b90 con 0x7f1aa01041a0 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:42.786 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38936/0 (socket says 172.21.15.174:38936) 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.123+0000 7f1a9f7fe700 1 -- 172.21.15.174:0/2975707819 learned_addr learned my addr 172.21.15.174:0/2975707819 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1a9f7fe700 1 -- 172.21.15.174:0/2975707819 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0107130 con 0x7f1aa01041a0 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1a9f7fe700 1 --2- 172.21.15.174:0/2975707819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa01045c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a900098a0 tx=0x7f1a90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1d4a8b09a01d66 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1a9e7fc700 1 -- 172.21.15.174:0/2975707819 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a90004160 con 0x7f1aa01041a0 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1a9e7fc700 1 -- 172.21.15.174:0/2975707819 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f1a900042c0 con 0x7f1aa01041a0 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1a9e7fc700 1 -- 172.21.15.174:0/2975707819 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a900119b0 con 0x7f1aa01041a0 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/2975707819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 msgr2=0x7f1aa01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 --2- 172.21.15.174:0/2975707819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa01045c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a900098a0 tx=0x7f1a90009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/2975707819 shutdown_connections 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 --2- 172.21.15.174:0/2975707819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa01045c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/2975707819 >> 172.21.15.174:0/2975707819 conn(0x7f1aa00ffb30 msgr2=0x7f1aa0101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/2975707819 shutdown_connections 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.124+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/2975707819 wait complete. 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1aa5ea6700 1 Processor -- start 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1aa5ea6700 1 -- start start 2024-02-10T06:38:42.787 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1aa5ea6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1aa5ea6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019a080 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1a9f7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa0199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38940/0 (socket says 172.21.15.174:38940) 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.125+0000 7f1a9f7fe700 1 -- 172.21.15.174:0/95643269 learned_addr learned my addr 172.21.15.174:0/95643269 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1a9f7fe700 1 -- 172.21.15.174:0/95643269 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90009550 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1a9f7fe700 1 --2- 172.21.15.174:0/95643269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa0199b40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1a900046c0 tx=0x7f1a90011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a900038b0 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f1a90003a10 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a90019610 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa019a280 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.126+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa019a720 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.127+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 2) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f1a900198d0 con 0x7f1aa01041a0 2024-02-10T06:38:42.788 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.127+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0063350 con 0x7f1aa01041a0 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.127+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1a9004cce0 con 0x7f1aa01041a0 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.129+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1a90019b80 con 0x7f1aa01041a0 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.199+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1aa019d1e0 con 0x7f1aa01041a0 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.199+0000 7f1a9cff9700 1 -- 172.21.15.174:0/95643269 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f1a90040e60 con 0x7f1aa01041a0 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 msgr2=0x7f1aa0199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 --2- 172.21.15.174:0/95643269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa0199b40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1a900046c0 tx=0x7f1a90011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 shutdown_connections 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 --2- 172.21.15.174:0/95643269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa01041a0 0x7f1aa0199b40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 >> 172.21.15.174:0/95643269 conn(0x7f1aa00ffb30 msgr2=0x7f1aa0100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 shutdown_connections 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:42.200+0000 7f1aa5ea6700 1 -- 172.21.15.174:0/95643269 wait complete. 2024-02-10T06:38:42.789 INFO:teuthology.orchestra.run.smithi174.stdout:mgr not available, waiting (3/15)... 2024-02-10T06:38:45.015 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:44 smithi174 ceph-mon[29878]: mgrmap e4: y(active, since 2s) 2024-02-10T06:38:45.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2909787503' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T06:38:46.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:46.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:38:46.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsid": "df845a14-c7de-11ee-95b8-87774f69a715", 2024-02-10T06:38:46.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 0 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T06:38:46.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T06:38:46.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ], 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T06:38:25.650575+0000", 2024-02-10T06:38:46.142 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout }, 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe21913b700 1 Processor -- start 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe21913b700 1 -- start start 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe21913b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe21913b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141067c0 con 0x7fe214105e60 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe212d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe212d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38944/0 (socket says 172.21.15.174:38944) 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe212d9d700 1 -- 172.21.15.174:0/1105458517 learned_addr learned my addr 172.21.15.174:0/1105458517 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe212d9d700 1 -- 172.21.15.174:0/1105458517 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe214106900 con 0x7fe214105e60 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.239+0000 7fe212d9d700 1 --2- 172.21.15.174:0/1105458517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214106280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098a0 tx=0x7fe204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8059dbbd156d8ad5 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe211d9b700 1 -- 172.21.15.174:0/1105458517 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe204004160 con 0x7fe214105e60 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe211d9b700 1 -- 172.21.15.174:0/1105458517 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fe2040042c0 con 0x7fe214105e60 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe211d9b700 1 -- 172.21.15.174:0/1105458517 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2040119b0 con 0x7fe214105e60 2024-02-10T06:38:46.143 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 -- 172.21.15.174:0/1105458517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 msgr2=0x7fe214106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 --2- 172.21.15.174:0/1105458517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214106280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098a0 tx=0x7fe204009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 -- 172.21.15.174:0/1105458517 shutdown_connections 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 --2- 172.21.15.174:0/1105458517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214106280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 -- 172.21.15.174:0/1105458517 >> 172.21.15.174:0/1105458517 conn(0x7fe2141014d0 msgr2=0x7fe2141038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 -- 172.21.15.174:0/1105458517 shutdown_connections 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.240+0000 7fe21913b700 1 -- 172.21.15.174:0/1105458517 wait complete. 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe21913b700 1 Processor -- start 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe21913b700 1 -- start start 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe21913b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe21913b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21419a070 con 0x7fe214105e60 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe212d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe212d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38954/0 (socket says 172.21.15.174:38954) 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.241+0000 7fe212d9d700 1 -- 172.21.15.174:0/2909787503 learned_addr learned my addr 172.21.15.174:0/2909787503 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe212d9d700 1 -- 172.21.15.174:0/2909787503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009550 con 0x7fe214105e60 2024-02-10T06:38:46.144 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe212d9d700 1 --2- 172.21.15.174:0/2909787503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214199b30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2040046c0 tx=0x7fe204011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2040038b0 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fe204003a10 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21419a270 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe204019610 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.242+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21419a710 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.243+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 4) v1 ==== 45886+0+0 (secure 0 0 0) 0x7fe204019880 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.243+0000 7fe1fbfff700 1 --2- 172.21.15.174:0/2909787503 >> v2:172.21.15.174:6800/2787410738 conn(0x7fe1fc038e50 0x7fe1fc03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.243+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe20404e130 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.243+0000 7fe21259c700 1 --2- 172.21.15.174:0/2909787503 >> v2:172.21.15.174:6800/2787410738 conn(0x7fe1fc038e50 0x7fe1fc03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.243+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214063350 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.243+0000 7fe21259c700 1 --2- 172.21.15.174:0/2909787503 >> v2:172.21.15.174:6800/2787410738 conn(0x7fe1fc038e50 0x7fe1fc03b310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.248+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe204018350 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.520+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe214107d10 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.520+0000 7fe1fbfff700 1 -- 172.21.15.174:0/2909787503 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fe20404b020 con 0x7fe214105e60 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 >> v2:172.21.15.174:6800/2787410738 conn(0x7fe1fc038e50 msgr2=0x7fe1fc03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:46.145 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 --2- 172.21.15.174:0/2909787503 >> v2:172.21.15.174:6800/2787410738 conn(0x7fe1fc038e50 0x7fe1fc03b310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208006e40 comp rx=0 tx=0).stop 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 msgr2=0x7fe214199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 --2- 172.21.15.174:0/2909787503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214199b30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2040046c0 tx=0x7fe204011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 shutdown_connections 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 --2- 172.21.15.174:0/2909787503 >> v2:172.21.15.174:6800/2787410738 conn(0x7fe1fc038e50 0x7fe1fc03b310 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 --2- 172.21.15.174:0/2909787503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe214105e60 0x7fe214199b30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 >> 172.21.15.174:0/2909787503 conn(0x7fe2141014d0 msgr2=0x7fe2141020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 shutdown_connections 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:45.524+0000 7fe21913b700 1 -- 172.21.15.174:0/2909787503 wait complete. 2024-02-10T06:38:46.146 INFO:teuthology.orchestra.run.smithi174.stdout:mgr is available 2024-02-10T06:38:47.065 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1767067498' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T06:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T06:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout fsid = df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.563+0000 7f0d83d69700 1 Processor -- start 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.563+0000 7f0d83d69700 1 -- start start 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.563+0000 7f0d83d69700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c079b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.563+0000 7f0d83d69700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c07a160 con 0x7f0d7c07b730 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.563+0000 7f0d81b05700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.563+0000 7f0d81b05700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48338/0 (socket says 172.21.15.174:48338) 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d81b05700 1 -- 172.21.15.174:0/1427853890 learned_addr learned my addr 172.21.15.174:0/1427853890 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d81b05700 1 -- 172.21.15.174:0/1427853890 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c07a2a0 con 0x7f0d7c07b730 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d81b05700 1 --2- 172.21.15.174:0/1427853890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c079b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0d780098a0 tx=0x7f0d78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4447afc86be7ec5 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d80b03700 1 -- 172.21.15.174:0/1427853890 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d78004160 con 0x7f0d7c07b730 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d80b03700 1 -- 172.21.15.174:0/1427853890 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f0d780042c0 con 0x7f0d7c07b730 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d80b03700 1 -- 172.21.15.174:0/1427853890 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d780119b0 con 0x7f0d7c07b730 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d83d69700 1 -- 172.21.15.174:0/1427853890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 msgr2=0x7f0d7c079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.564+0000 7f0d83d69700 1 --2- 172.21.15.174:0/1427853890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c079b90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0d780098a0 tx=0x7f0d78009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 -- 172.21.15.174:0/1427853890 shutdown_connections 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 --2- 172.21.15.174:0/1427853890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c079b90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 -- 172.21.15.174:0/1427853890 >> 172.21.15.174:0/1427853890 conn(0x7f0d7c1014d0 msgr2=0x7f0d7c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 -- 172.21.15.174:0/1427853890 shutdown_connections 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 -- 172.21.15.174:0/1427853890 wait complete. 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 Processor -- start 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 -- start start 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d83d69700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c19c250 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.565+0000 7f0d81b05700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d81b05700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48348/0 (socket says 172.21.15.174:48348) 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d81b05700 1 -- 172.21.15.174:0/1767067498 learned_addr learned my addr 172.21.15.174:0/1767067498 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d81b05700 1 -- 172.21.15.174:0/1767067498 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d78009550 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d81b05700 1 --2- 172.21.15.174:0/1767067498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c19bd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006af0 tx=0x7f0d78011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d780039e0 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f0d78003b40 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d78019700 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c19c450 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.566+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c19c8f0 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.567+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 4) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f0d78019880 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.567+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c10c1b0 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.567+0000 7f0d72ffd700 1 --2- 172.21.15.174:0/1767067498 >> v2:172.21.15.174:6800/2787410738 conn(0x7f0d68038ea0 0x7f0d6803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.567+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0d7804cfb0 con 0x7f0d7c07b730 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.567+0000 7f0d81304700 1 --2- 172.21.15.174:0/1767067498 >> v2:172.21.15.174:6800/2787410738 conn(0x7f0d68038ea0 0x7f0d6803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.568+0000 7f0d81304700 1 --2- 172.21.15.174:0/1767067498 >> v2:172.21.15.174:6800/2787410738 conn(0x7f0d68038ea0 0x7f0d6803b360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c006fd0 tx=0x7f0d6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.572+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d78019b30 con 0x7f0d7c07b730 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.763+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0d7c10c3c0 con 0x7f0d7c07b730 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.764+0000 7f0d72ffd700 1 -- 172.21.15.174:0/1767067498 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7f0d78010380 con 0x7f0d7c07b730 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.766+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 >> v2:172.21.15.174:6800/2787410738 conn(0x7f0d68038ea0 msgr2=0x7f0d6803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.766+0000 7f0d83d69700 1 --2- 172.21.15.174:0/1767067498 >> v2:172.21.15.174:6800/2787410738 conn(0x7f0d68038ea0 0x7f0d6803b360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c006fd0 tx=0x7f0d6c006e40 comp rx=0 tx=0).stop 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.766+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 msgr2=0x7f0d7c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.766+0000 7f0d83d69700 1 --2- 172.21.15.174:0/1767067498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c19bd10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006af0 tx=0x7f0d78011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.767+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 shutdown_connections 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.767+0000 7f0d83d69700 1 --2- 172.21.15.174:0/1767067498 >> v2:172.21.15.174:6800/2787410738 conn(0x7f0d68038ea0 0x7f0d6803b360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.767+0000 7f0d83d69700 1 --2- 172.21.15.174:0/1767067498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d7c07b730 0x7f0d7c19bd10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.767+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 >> 172.21.15.174:0/1767067498 conn(0x7f0d7c1014d0 msgr2=0x7f0d7c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.767+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 shutdown_connections 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:46.767+0000 7f0d83d69700 1 -- 172.21.15.174:0/1767067498 wait complete. 2024-02-10T06:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout:Enabling cephadm module... 2024-02-10T06:38:49.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/948920873' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-10T06:38:49.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ignoring --setuser ceph since I am not root 2024-02-10T06:38:49.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ignoring --setgroup ceph since I am not root 2024-02-10T06:38:49.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:49.298+0000 7f46995cc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:38:49.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:49.389+0000 7f46995cc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:38:49.545 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f406a602700 1 Processor -- start 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f406a602700 1 -- start start 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f406a602700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f406a602700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40641067c0 con 0x7f4064105e60 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f4063fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f4063fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48364/0 (socket says 172.21.15.174:48364) 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f4063fff700 1 -- 172.21.15.174:0/13646766 learned_addr learned my addr 172.21.15.174:0/13646766 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f4063fff700 1 -- 172.21.15.174:0/13646766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064106900 con 0x7f4064105e60 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.747+0000 7f4063fff700 1 --2- 172.21.15.174:0/13646766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064106280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f40580098a0 tx=0x7f4058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700c4dc12f59f7f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f4062ffd700 1 -- 172.21.15.174:0/13646766 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4058004160 con 0x7f4064105e60 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f4062ffd700 1 -- 172.21.15.174:0/13646766 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f40580042c0 con 0x7f4064105e60 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f4062ffd700 1 -- 172.21.15.174:0/13646766 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40580119b0 con 0x7f4064105e60 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 -- 172.21.15.174:0/13646766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 msgr2=0x7f4064106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 --2- 172.21.15.174:0/13646766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064106280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f40580098a0 tx=0x7f4058009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 -- 172.21.15.174:0/13646766 shutdown_connections 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 --2- 172.21.15.174:0/13646766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064106280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 -- 172.21.15.174:0/13646766 >> 172.21.15.174:0/13646766 conn(0x7f40641014d0 msgr2=0x7f40641038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 -- 172.21.15.174:0/13646766 shutdown_connections 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.748+0000 7f406a602700 1 -- 172.21.15.174:0/13646766 wait complete. 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f406a602700 1 Processor -- start 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f406a602700 1 -- start start 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f406a602700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f406a602700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406419a070 con 0x7f4064105e60 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f4063fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f4063fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48366/0 (socket says 172.21.15.174:48366) 2024-02-10T06:38:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f4063fff700 1 -- 172.21.15.174:0/948920873 learned_addr learned my addr 172.21.15.174:0/948920873 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.749+0000 7f4063fff700 1 -- 172.21.15.174:0/948920873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4058009550 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.750+0000 7f4063fff700 1 --2- 172.21.15.174:0/948920873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064199b30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f40580046c0 tx=0x7f4058011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.750+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40580038b0 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.750+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f4058003a10 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.750+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4058019610 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.750+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406419a270 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.750+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406419a710 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.751+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 4) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f4058019880 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.751+0000 7f40617fa700 1 --2- 172.21.15.174:0/948920873 >> v2:172.21.15.174:6800/2787410738 conn(0x7f404c038e50 0x7f404c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.751+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4064063350 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.751+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f405804d4d0 con 0x7f4064105e60 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.751+0000 7f40637fe700 1 --2- 172.21.15.174:0/948920873 >> v2:172.21.15.174:6800/2787410738 conn(0x7f404c038e50 0x7f404c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.751+0000 7f40637fe700 1 --2- 172.21.15.174:0/948920873 >> v2:172.21.15.174:6800/2787410738 conn(0x7f404c038e50 0x7f404c03b310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4054006fd0 tx=0x7f4054006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:49.548 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:47.756+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4058018350 con 0x7f4064105e60 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.002+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f406419d100 con 0x7f4064105e60 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.745+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f4058023c20 con 0x7f4064105e60 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.746+0000 7f40617fa700 1 -- 172.21.15.174:0/948920873 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mgrmap(e 5) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f405802d950 con 0x7f4064105e60 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.748+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 >> v2:172.21.15.174:6800/2787410738 conn(0x7f404c038e50 msgr2=0x7f404c03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.748+0000 7f406a602700 1 --2- 172.21.15.174:0/948920873 >> v2:172.21.15.174:6800/2787410738 conn(0x7f404c038e50 0x7f404c03b310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4054006fd0 tx=0x7f4054006e40 comp rx=0 tx=0).stop 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.748+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 msgr2=0x7f4064199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.748+0000 7f406a602700 1 --2- 172.21.15.174:0/948920873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064199b30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f40580046c0 tx=0x7f4058011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.749+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 shutdown_connections 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.749+0000 7f406a602700 1 --2- 172.21.15.174:0/948920873 >> v2:172.21.15.174:6800/2787410738 conn(0x7f404c038e50 0x7f404c03b310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.749+0000 7f406a602700 1 --2- 172.21.15.174:0/948920873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064105e60 0x7f4064199b30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.749+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 >> 172.21.15.174:0/948920873 conn(0x7f40641014d0 msgr2=0x7f40641020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.749+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 shutdown_connections 2024-02-10T06:38:49.549 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:48.749+0000 7f406a602700 1 -- 172.21.15.174:0/948920873 wait complete. 2024-02-10T06:38:49.791 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/948920873' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-10T06:38:49.791 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:49 smithi174 ceph-mon[29878]: mgrmap e5: y(active, since 7s) 2024-02-10T06:38:50.152 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.058+0000 7f46995cc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:38:50.152 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.151+0000 7f46995cc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:38:50.474 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.297+0000 7f46995cc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:38:50.474 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.391+0000 7f46995cc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:38:50.474 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.473+0000 7f46995cc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:38:50.945 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:38:50.945 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-10T06:38:50.945 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T06:38:50.945 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T06:38:50.945 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T06:38:50.945 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a23be0700 1 Processor -- start 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a23be0700 1 -- start start 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a23be0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c104b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a23be0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0796c0 con 0x7f5a1c104740 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a2197c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c104b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a2197c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c104b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48386/0 (socket says 172.21.15.174:48386) 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a2197c700 1 -- 172.21.15.174:0/1656941846 learned_addr learned my addr 172.21.15.174:0/1656941846 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.000+0000 7f5a2197c700 1 -- 172.21.15.174:0/1656941846 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c1050a0 con 0x7f5a1c104740 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a2197c700 1 --2- 172.21.15.174:0/1656941846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c104b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb54e52c78646a server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a2097a700 1 -- 172.21.15.174:0/1656941846 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a18004160 con 0x7f5a1c104740 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a2097a700 1 -- 172.21.15.174:0/1656941846 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5a180042c0 con 0x7f5a1c104740 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a2097a700 1 -- 172.21.15.174:0/1656941846 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a180119b0 con 0x7f5a1c104740 2024-02-10T06:38:50.946 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a23be0700 1 -- 172.21.15.174:0/1656941846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 msgr2=0x7f5a1c104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a23be0700 1 --2- 172.21.15.174:0/1656941846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c104b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a18009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a23be0700 1 -- 172.21.15.174:0/1656941846 shutdown_connections 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a23be0700 1 --2- 172.21.15.174:0/1656941846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c104b60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.001+0000 7f5a23be0700 1 -- 172.21.15.174:0/1656941846 >> 172.21.15.174:0/1656941846 conn(0x7f5a1c100300 msgr2=0x7f5a1c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.002+0000 7f5a23be0700 1 -- 172.21.15.174:0/1656941846 shutdown_connections 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.002+0000 7f5a23be0700 1 -- 172.21.15.174:0/1656941846 wait complete. 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.002+0000 7f5a23be0700 1 Processor -- start 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.002+0000 7f5a23be0700 1 -- start start 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a23be0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a23be0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c19c210 con 0x7f5a1c104740 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a2197c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a2197c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48394/0 (socket says 172.21.15.174:48394) 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a2197c700 1 -- 172.21.15.174:0/3542137941 learned_addr learned my addr 172.21.15.174:0/3542137941 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a2197c700 1 -- 172.21.15.174:0/3542137941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a18009550 con 0x7f5a1c104740 2024-02-10T06:38:50.947 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a2197c700 1 --2- 172.21.15.174:0/3542137941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c19bcd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a18011670 tx=0x7f5a18011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a18003af0 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f5a1801a8c0 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c19c410 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a18019730 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.003+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c19c8b0 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.004+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f5a180199f0 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.004+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c1959c0 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.004+0000 7f5a12ffd700 1 --2- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 0x7f5a0803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.005+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5a1804d180 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.005+0000 7f5a2117b700 1 -- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 msgr2=0x7f5a0803b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.005+0000 7f5a2117b700 1 --2- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 0x7f5a0803b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.009+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a18018c70 con 0x7f5a1c104740 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.206+0000 7f5a2117b700 1 -- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 msgr2=0x7f5a0803b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:50.948 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.206+0000 7f5a2117b700 1 --2- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 0x7f5a0803b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.261+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5a1c105e30 con 0x7f5a1c104740 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.261+0000 7f5a12ffd700 1 -- 172.21.15.174:0/3542137941 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5a18018860 con 0x7f5a1c104740 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 msgr2=0x7f5a0803b360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 --2- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 0x7f5a0803b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 msgr2=0x7f5a1c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 --2- 172.21.15.174:0/3542137941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c19bcd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a18011670 tx=0x7f5a18011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 shutdown_connections 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 --2- 172.21.15.174:0/3542137941 >> v2:172.21.15.174:6800/2787410738 conn(0x7f5a08038ea0 0x7f5a0803b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 --2- 172.21.15.174:0/3542137941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1c104740 0x7f5a1c19bcd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 >> 172.21.15.174:0/3542137941 conn(0x7f5a1c100300 msgr2=0x7f5a1c192b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 shutdown_connections 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:50.264+0000 7f5a23be0700 1 -- 172.21.15.174:0/3542137941 wait complete. 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for the mgr to restart... 2024-02-10T06:38:50.949 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for mgr epoch 5... 2024-02-10T06:38:51.034 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3542137941' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T06:38:51.034 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.740+0000 7f46995cc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:38:51.034 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.849+0000 7f46995cc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:38:51.034 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:50 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:50.933+0000 7f46995cc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:38:51.034 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:51 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:51.033+0000 7f46995cc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:38:51.286 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:51 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:51.214+0000 7f46995cc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:38:52.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:52.172+0000 7f46995cc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:38:52.633 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:52.300+0000 7f46995cc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:38:52.633 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:52.632+0000 7f46995cc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:38:52.916 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:52.722+0000 7f46995cc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:38:52.917 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:52.916+0000 7f46995cc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:38:53.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:53.099+0000 7f46995cc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:38:53.640 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:53.380+0000 7f46995cc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:38:53.641 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:53.639+0000 7f46995cc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:38:53.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:53.871+0000 7f46995cc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:38:54.941 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:54 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:54.665+0000 7f46995cc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:38:55.589 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:55.266+0000 7f46995cc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:38:55.589 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:55.367+0000 7f46995cc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:38:55.590 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:38:55.588+0000 7f46995cc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:38:55.891 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: Active manager daemon y restarted 2024-02-10T06:38:55.891 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: Activating manager daemon y 2024-02-10T06:38:55.891 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: osdmap e2: 0 total, 0 up, 0 in 2024-02-10T06:38:55.891 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: mgrmap e6: y(active, starting, since 0.00357965s) 2024-02-10T06:38:55.891 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:38:55.891 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: Manager daemon y is now available 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:38:55.892 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:55 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:38:56.926 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:56 smithi174 ceph-mon[29878]: Found migration_current of "None". Setting to last migration. 2024-02-10T06:38:56.926 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:56 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:38:56.927 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:56 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:38:56.927 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:56 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:38:56.927 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:56 smithi174 ceph-mon[29878]: mgrmap e7: y(active, since 1.00617s) 2024-02-10T06:38:57.212 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:38:57.220 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-10T06:38:57.220 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf3bd00700 1 Processor -- start 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf3bd00700 1 -- start start 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf3bd00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf34104ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf3bd00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34079740 con 0x7fdf341046c0 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf39a9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf34104ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf39a9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf34104ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48406/0 (socket says 172.21.15.174:48406) 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.381+0000 7fdf39a9c700 1 -- 172.21.15.174:0/1813529228 learned_addr learned my addr 172.21.15.174:0/1813529228 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf39a9c700 1 -- 172.21.15.174:0/1813529228 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34105020 con 0x7fdf341046c0 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf39a9c700 1 --2- 172.21.15.174:0/1813529228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf34104ae0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdf300098a0 tx=0x7fdf30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237ff9193f7673f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf38a9a700 1 -- 172.21.15.174:0/1813529228 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf30004160 con 0x7fdf341046c0 2024-02-10T06:38:57.221 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf38a9a700 1 -- 172.21.15.174:0/1813529228 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fdf300042c0 con 0x7fdf341046c0 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf38a9a700 1 -- 172.21.15.174:0/1813529228 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf300119b0 con 0x7fdf341046c0 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf3bd00700 1 -- 172.21.15.174:0/1813529228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 msgr2=0x7fdf34104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.382+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/1813529228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf34104ae0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdf300098a0 tx=0x7fdf30009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 -- 172.21.15.174:0/1813529228 shutdown_connections 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/1813529228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf34104ae0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 -- 172.21.15.174:0/1813529228 >> 172.21.15.174:0/1813529228 conn(0x7fdf34100300 msgr2=0x7fdf34102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 -- 172.21.15.174:0/1813529228 shutdown_connections 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 -- 172.21.15.174:0/1813529228 wait complete. 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 Processor -- start 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.383+0000 7fdf3bd00700 1 -- start start 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf3bd00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf3419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf3bd00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3419c200 con 0x7fdf341046c0 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf39a9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf3419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:57.222 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf39a9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf3419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48416/0 (socket says 172.21.15.174:48416) 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf39a9c700 1 -- 172.21.15.174:0/213498407 learned_addr learned my addr 172.21.15.174:0/213498407 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf39a9c700 1 -- 172.21.15.174:0/213498407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30009550 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf39a9c700 1 --2- 172.21.15.174:0/213498407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf3419bcc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf300046c0 tx=0x7fdf30011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf300038b0 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fdf30003a10 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.384+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3419c400 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.385+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf30019690 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.385+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3419c8a0 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.385+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fdf30019870 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.386+0000 7fdf2affd700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.386+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf30052170 con 0x7fdf341046c0 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.386+0000 7fdf3929b700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 msgr2=0x7fdf2003b000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.386+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:38:57.223 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.386+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 --> v2:172.21.15.174:6800/2787410738 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf34105db0 con 0x7fdf20038b40 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.587+0000 7fdf3929b700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 msgr2=0x7fdf2003b000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.587+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.988+0000 7fdf3929b700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 msgr2=0x7fdf2003b000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:51.988+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:52.789+0000 7fdf3929b700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 msgr2=0x7fdf2003b000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:52.789+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:54.390+0000 7fdf3929b700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 msgr2=0x7fdf2003b000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2787410738 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:54.390+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:55.604+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7fdf3000b520 con 0x7fdf341046c0 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:55.604+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 msgr2=0x7fdf2003b000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:55.604+0000 7fdf2affd700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.224 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.606+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mgrmap(e 7) v1 ==== 45791+0+0 (secure 0 0 0) 0x7fdf30043160 con 0x7fdf341046c0 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.606+0000 7fdf2affd700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/1048754011 conn(0x7fdf2003c320 0x7fdf2003e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.606+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 --> v2:172.21.15.174:6800/1048754011 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf34105db0 con 0x7fdf2003c320 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.606+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/1048754011 conn(0x7fdf2003c320 0x7fdf2003e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.607+0000 7fdf3929b700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/1048754011 conn(0x7fdf2003c320 0x7fdf2003e710 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf24003a10 tx=0x7fdf240092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.607+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdf34105db0 con 0x7fdf2003c320 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 --> v2:172.21.15.174:6800/1048754011 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdf34105f20 con 0x7fdf2003c320 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf2affd700 1 -- 172.21.15.174:0/213498407 <== mgr.14120 v2:172.21.15.174:6800/1048754011 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdf34105f20 con 0x7fdf2003c320 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/1048754011 conn(0x7fdf2003c320 msgr2=0x7fdf2003e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/1048754011 conn(0x7fdf2003c320 0x7fdf2003e710 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf24003a10 tx=0x7fdf240092b0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 msgr2=0x7fdf3419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/213498407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf3419bcc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf300046c0 tx=0x7fdf30011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 shutdown_connections 2024-02-10T06:38:57.225 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/1048754011 conn(0x7fdf2003c320 0x7fdf2003e710 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/213498407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf341046c0 0x7fdf3419bcc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 --2- 172.21.15.174:0/213498407 >> v2:172.21.15.174:6800/2787410738 conn(0x7fdf20038b40 0x7fdf2003b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.614+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 >> 172.21.15.174:0/213498407 conn(0x7fdf34100300 msgr2=0x7fdf34192b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.615+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 shutdown_connections 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:56.615+0000 7fdf3bd00700 1 -- 172.21.15.174:0/213498407 wait complete. 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:mgr epoch 5 is available 2024-02-10T06:38:57.226 INFO:teuthology.orchestra.run.smithi174.stdout:Setting orchestrator backend to cephadm... 2024-02-10T06:38:57.926 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: [10/Feb/2024:06:38:56] ENGINE Bus STARTING 2024-02-10T06:38:57.926 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T06:38:57.926 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T06:38:57.926 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: [10/Feb/2024:06:38:56] ENGINE Serving on https://172.21.15.174:7150 2024-02-10T06:38:57.927 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: [10/Feb/2024:06:38:56] ENGINE Serving on http://172.21.15.174:8765 2024-02-10T06:38:57.927 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: [10/Feb/2024:06:38:56] ENGINE Bus STARTED 2024-02-10T06:38:57.927 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:57 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff3898ec700 1 Processor -- start 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff3898ec700 1 -- start start 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff3898ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff384105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff3898ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841056f0 con 0x7ff384104d00 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff382ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff384105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff382ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff384105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53420/0 (socket says 172.21.15.174:53420) 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff382ffd700 1 -- 172.21.15.174:0/1992043928 learned_addr learned my addr 172.21.15.174:0/1992043928 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.707+0000 7ff382ffd700 1 -- 172.21.15.174:0/1992043928 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff384105f00 con 0x7ff384104d00 2024-02-10T06:38:58.523 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff382ffd700 1 --2- 172.21.15.174:0/1992043928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff384105120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff3740098a0 tx=0x7ff374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eda901942eb152c server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff381ffb700 1 -- 172.21.15.174:0/1992043928 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff374004160 con 0x7ff384104d00 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff381ffb700 1 -- 172.21.15.174:0/1992043928 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7ff3740042c0 con 0x7ff384104d00 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff381ffb700 1 -- 172.21.15.174:0/1992043928 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3740119b0 con 0x7ff384104d00 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff3898ec700 1 -- 172.21.15.174:0/1992043928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 msgr2=0x7ff384105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff3898ec700 1 --2- 172.21.15.174:0/1992043928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff384105120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff3740098a0 tx=0x7ff374009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff3898ec700 1 -- 172.21.15.174:0/1992043928 shutdown_connections 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff3898ec700 1 --2- 172.21.15.174:0/1992043928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff384105120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff3898ec700 1 -- 172.21.15.174:0/1992043928 >> 172.21.15.174:0/1992043928 conn(0x7ff3841002c0 msgr2=0x7ff3841026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.708+0000 7ff3898ec700 1 -- 172.21.15.174:0/1992043928 shutdown_connections 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.709+0000 7ff3898ec700 1 -- 172.21.15.174:0/1992043928 wait complete. 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.709+0000 7ff3898ec700 1 Processor -- start 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.709+0000 7ff3898ec700 1 -- start start 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.709+0000 7ff3898ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff3841989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:58.524 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.709+0000 7ff3898ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff384198ee0 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff382ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff3841989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff382ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff3841989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53424/0 (socket says 172.21.15.174:53424) 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff382ffd700 1 -- 172.21.15.174:0/3766390652 learned_addr learned my addr 172.21.15.174:0/3766390652 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff382ffd700 1 -- 172.21.15.174:0/3766390652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff374009550 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff382ffd700 1 --2- 172.21.15.174:0/3766390652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff3841989a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3740046c0 tx=0x7ff374011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3740039c0 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7ff374003b20 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3841990e0 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff37401a780 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.710+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff384199580 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.711+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7ff37401a8e0 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.711+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff384063350 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.711+0000 7ff3888ea700 1 --2- 172.21.15.174:0/3766390652 >> v2:172.21.15.174:6800/1048754011 conn(0x7ff36c038ea0 0x7ff36c03b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.711+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7ff37404ccf0 con 0x7ff384104d00 2024-02-10T06:38:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.711+0000 7ff3827fc700 1 --2- 172.21.15.174:0/3766390652 >> v2:172.21.15.174:6800/1048754011 conn(0x7ff36c038ea0 0x7ff36c03b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.712+0000 7ff3827fc700 1 --2- 172.21.15.174:0/3766390652 >> v2:172.21.15.174:6800/1048754011 conn(0x7ff36c038ea0 0x7ff36c03b360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff378006fd0 tx=0x7ff378006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.716+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff37401ab90 con 0x7ff384104d00 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.925+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7ff384101540 con 0x7ff36c038ea0 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.930+0000 7ff3888ea700 1 -- 172.21.15.174:0/3766390652 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff384101540 con 0x7ff36c038ea0 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 >> v2:172.21.15.174:6800/1048754011 conn(0x7ff36c038ea0 msgr2=0x7ff36c03b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 --2- 172.21.15.174:0/3766390652 >> v2:172.21.15.174:6800/1048754011 conn(0x7ff36c038ea0 0x7ff36c03b360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff378006fd0 tx=0x7ff378006e40 comp rx=0 tx=0).stop 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 msgr2=0x7ff3841989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 --2- 172.21.15.174:0/3766390652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff3841989a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3740046c0 tx=0x7ff374011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 shutdown_connections 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 --2- 172.21.15.174:0/3766390652 >> v2:172.21.15.174:6800/1048754011 conn(0x7ff36c038ea0 0x7ff36c03b360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 --2- 172.21.15.174:0/3766390652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff384104d00 0x7ff3841989a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 >> 172.21.15.174:0/3766390652 conn(0x7ff3841002c0 msgr2=0x7ff384100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 shutdown_connections 2024-02-10T06:38:58.526 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:57.932+0000 7ff3898ec700 1 -- 172.21.15.174:0/3766390652 wait complete. 2024-02-10T06:38:58.770 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:58 smithi174 ceph-mon[29878]: mgrmap e8: y(active, since 2s) 2024-02-10T06:38:58.770 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:58 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:38:58.770 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:58 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:38:59.788 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:38:59 smithi174 ceph-mon[29878]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb31a3a9700 1 Processor -- start 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb31a3a9700 1 -- start start 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb31a3a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb3141051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb31a3a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3141057a0 con 0x7fb314104db0 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb313fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb3141051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb313fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb3141051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53430/0 (socket says 172.21.15.174:53430) 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.002+0000 7fb313fff700 1 -- 172.21.15.174:0/2230626672 learned_addr learned my addr 172.21.15.174:0/2230626672 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb313fff700 1 -- 172.21.15.174:0/2230626672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314105fb0 con 0x7fb314104db0 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb313fff700 1 --2- 172.21.15.174:0/2230626672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb3141051d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb3040098a0 tx=0x7fb304009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c5262d1787c1ab server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb312ffd700 1 -- 172.21.15.174:0/2230626672 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb304004160 con 0x7fb314104db0 2024-02-10T06:38:59.796 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb312ffd700 1 -- 172.21.15.174:0/2230626672 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb3040042c0 con 0x7fb314104db0 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb312ffd700 1 -- 172.21.15.174:0/2230626672 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3040119b0 con 0x7fb314104db0 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb31a3a9700 1 -- 172.21.15.174:0/2230626672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 msgr2=0x7fb3141051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb31a3a9700 1 --2- 172.21.15.174:0/2230626672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb3141051d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb3040098a0 tx=0x7fb304009bb0 comp rx=0 tx=0).stop 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb31a3a9700 1 -- 172.21.15.174:0/2230626672 shutdown_connections 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb31a3a9700 1 --2- 172.21.15.174:0/2230626672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb3141051d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.003+0000 7fb31a3a9700 1 -- 172.21.15.174:0/2230626672 >> 172.21.15.174:0/2230626672 conn(0x7fb3141003b0 msgr2=0x7fb3141027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.004+0000 7fb31a3a9700 1 -- 172.21.15.174:0/2230626672 shutdown_connections 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.004+0000 7fb31a3a9700 1 -- 172.21.15.174:0/2230626672 wait complete. 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.004+0000 7fb31a3a9700 1 Processor -- start 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.004+0000 7fb31a3a9700 1 -- start start 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.004+0000 7fb31a3a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb314199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.004+0000 7fb31a3a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31419a070 con 0x7fb314104db0 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb313fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb314199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb313fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb314199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53444/0 (socket says 172.21.15.174:53444) 2024-02-10T06:38:59.797 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb313fff700 1 -- 172.21.15.174:0/4040997636 learned_addr learned my addr 172.21.15.174:0/4040997636 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb313fff700 1 -- 172.21.15.174:0/4040997636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009550 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb313fff700 1 --2- 172.21.15.174:0/4040997636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb314199b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb304006af0 tx=0x7fb304011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3040039e0 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fb304003b40 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31419a270 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb304019700 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.005+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31419a710 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.006+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fb3040198a0 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.006+0000 7fb3117fa700 1 --2- 172.21.15.174:0/4040997636 >> v2:172.21.15.174:6800/1048754011 conn(0x7fb2fc038ef0 0x7fb2fc03b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.006+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fb30404ddc0 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.006+0000 7fb3137fe700 1 --2- 172.21.15.174:0/4040997636 >> v2:172.21.15.174:6800/1048754011 conn(0x7fb2fc038ef0 0x7fb2fc03b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.006+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb314193c80 con 0x7fb314104db0 2024-02-10T06:38:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.007+0000 7fb3137fe700 1 --2- 172.21.15.174:0/4040997636 >> v2:172.21.15.174:6800/1048754011 conn(0x7fb2fc038ef0 0x7fb2fc03b3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb308006fd0 tx=0x7fb308006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.011+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb304019b50 con 0x7fb314104db0 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.207+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fb314062160 con 0x7fb2fc038ef0 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.207+0000 7fb3117fa700 1 -- 172.21.15.174:0/4040997636 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fb314062160 con 0x7fb2fc038ef0 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 >> v2:172.21.15.174:6800/1048754011 conn(0x7fb2fc038ef0 msgr2=0x7fb2fc03b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 --2- 172.21.15.174:0/4040997636 >> v2:172.21.15.174:6800/1048754011 conn(0x7fb2fc038ef0 0x7fb2fc03b3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb308006fd0 tx=0x7fb308006e40 comp rx=0 tx=0).stop 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 msgr2=0x7fb314199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 --2- 172.21.15.174:0/4040997636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb314199b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb304006af0 tx=0x7fb304011a50 comp rx=0 tx=0).stop 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 shutdown_connections 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 --2- 172.21.15.174:0/4040997636 >> v2:172.21.15.174:6800/1048754011 conn(0x7fb2fc038ef0 0x7fb2fc03b3b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 --2- 172.21.15.174:0/4040997636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb314104db0 0x7fb314199b30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 >> 172.21.15.174:0/4040997636 conn(0x7fb3141003b0 msgr2=0x7fb314101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 shutdown_connections 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:38:59.210+0000 7fb31a3a9700 1 -- 172.21.15.174:0/4040997636 wait complete. 2024-02-10T06:38:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:Generating ssh key... 2024-02-10T06:39:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-mon[29878]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:00.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: Generating public/private rsa key pair. 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: Your identification has been saved in /tmp/tmpz7s_t1u6/key. 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: Your public key has been saved in /tmp/tmpz7s_t1u6/key.pub. 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: The key fingerprint is: 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: SHA256:7kz2nj0CNWlzLvcT+K+VqBzCkjLr0umXPzrZcA4jfOg ceph-df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: The key's randomart image is: 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: +---[RSA 3072]----+ 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | | 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | | 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | . | 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | * . | 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | . . So = . | 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | + *+.. + o .| 2024-02-10T06:39:00.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | oo+o&+ + + o.| 2024-02-10T06:39:00.741 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | . E+X.=+o+ +. | 2024-02-10T06:39:00.741 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: | ++..*o=+...+.| 2024-02-10T06:39:00.741 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:00 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: +----[SHA256]-----+ 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91ae335700 1 Processor -- start 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91ae335700 1 -- start start 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91ae335700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91ae335700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81067c0 con 0x7f91a8105e60 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53460/0 (socket says 172.21.15.174:53460) 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91a7fff700 1 -- 172.21.15.174:0/2546093582 learned_addr learned my addr 172.21.15.174:0/2546093582 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.257+0000 7f91a7fff700 1 -- 172.21.15.174:0/2546093582 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8106900 con 0x7f91a8105e60 2024-02-10T06:39:01.285 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91a7fff700 1 --2- 172.21.15.174:0/2546093582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8106280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91980098a0 tx=0x7f9198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac6057f320e42e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91a6ffd700 1 -- 172.21.15.174:0/2546093582 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9198004160 con 0x7f91a8105e60 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91a6ffd700 1 -- 172.21.15.174:0/2546093582 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f91980042c0 con 0x7f91a8105e60 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91a6ffd700 1 -- 172.21.15.174:0/2546093582 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91980119b0 con 0x7f91a8105e60 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 -- 172.21.15.174:0/2546093582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 msgr2=0x7f91a8106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 --2- 172.21.15.174:0/2546093582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8106280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91980098a0 tx=0x7f9198009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 -- 172.21.15.174:0/2546093582 shutdown_connections 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 --2- 172.21.15.174:0/2546093582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8106280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 -- 172.21.15.174:0/2546093582 >> 172.21.15.174:0/2546093582 conn(0x7f91a81014d0 msgr2=0x7f91a81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 -- 172.21.15.174:0/2546093582 shutdown_connections 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.258+0000 7f91ae335700 1 -- 172.21.15.174:0/2546093582 wait complete. 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91ae335700 1 Processor -- start 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91ae335700 1 -- start start 2024-02-10T06:39:01.286 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91ae335700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91ae335700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819a070 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53468/0 (socket says 172.21.15.174:53468) 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.259+0000 7f91a7fff700 1 -- 172.21.15.174:0/2360085155 learned_addr learned my addr 172.21.15.174:0/2360085155 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91a7fff700 1 -- 172.21.15.174:0/2360085155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9198009550 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91a7fff700 1 --2- 172.21.15.174:0/2360085155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8199b30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91980046c0 tx=0x7f9198011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91980038b0 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a819a270 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f9198003a10 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9198019610 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.260+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a819a710 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.261+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f91980198d0 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.261+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a8063350 con 0x7f91a8105e60 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.261+0000 7f91a57fa700 1 --2- 172.21.15.174:0/2360085155 >> v2:172.21.15.174:6800/1048754011 conn(0x7f9190038e50 0x7f919003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:01.287 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.261+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f919804cfb0 con 0x7f91a8105e60 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.261+0000 7f91a77fe700 1 --2- 172.21.15.174:0/2360085155 >> v2:172.21.15.174:6800/1048754011 conn(0x7f9190038e50 0x7f919003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.261+0000 7f91a77fe700 1 --2- 172.21.15.174:0/2360085155 >> v2:172.21.15.174:6800/1048754011 conn(0x7f9190038e50 0x7f919003b310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f919c006fd0 tx=0x7f919c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.266+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9198018350 con 0x7f91a8105e60 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.458+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f91a81027b0 con 0x7f9190038e50 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.646+0000 7f91a57fa700 1 -- 172.21.15.174:0/2360085155 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91a81027b0 con 0x7f9190038e50 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 >> v2:172.21.15.174:6800/1048754011 conn(0x7f9190038e50 msgr2=0x7f919003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 --2- 172.21.15.174:0/2360085155 >> v2:172.21.15.174:6800/1048754011 conn(0x7f9190038e50 0x7f919003b310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f919c006fd0 tx=0x7f919c006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 msgr2=0x7f91a8199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 --2- 172.21.15.174:0/2360085155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8199b30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91980046c0 tx=0x7f9198011a50 comp rx=0 tx=0).stop 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 shutdown_connections 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 --2- 172.21.15.174:0/2360085155 >> v2:172.21.15.174:6800/1048754011 conn(0x7f9190038e50 0x7f919003b310 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 --2- 172.21.15.174:0/2360085155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8105e60 0x7f91a8199b30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:01.288 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 >> 172.21.15.174:0/2360085155 conn(0x7f91a81014d0 msgr2=0x7f91a81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:01.289 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 shutdown_connections 2024-02-10T06:39:01.289 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:00.648+0000 7f91ae335700 1 -- 172.21.15.174:0/2360085155 wait complete. 2024-02-10T06:39:01.920 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:01 smithi174 ceph-mon[29878]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:01.920 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:01 smithi174 ceph-mon[29878]: Generating ssh key... 2024-02-10T06:39:02.499 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKXkgq4Zsitu3iChjhA6edpwDy5zmESu4bPYzqyR1WczQ8o2Y3PFgDMPCTmELJQH7A7mO3sUAYjBm830FGEzHkLROMjcvScUQ4wlbsvRDdAsTBiqy1oJIHjtrvBF76sFNfJGj6g8yAL+Msg+QJvZAcXyeye+OFuwSIE4VgvK3fdXeDE8lRNg7dSg6KPTzVYWEXrvWe5Ttg05SAn9gIBbRgx7D8Jm3Alh8vthW3ib9Exx/6mLE2YhvGi3T49Vtblp/5tL93a1rjpfABalMB/+vG24i+ZGQG/JnqBpi8ZS93OJCHN2v3v6xh1UG82QeS3aTuofPYTLsMDeA+gM7NTvb39WCeyJFGimPllri8KFGtLXXKnOCt+kGM22+dqHoCGcI5EaX93BWt8uwNEtAdaHkH0/yNOBveBzjg88LYUFXVenMrqkXnWmBLU6n2nwo6k/2Zcr/pKcyWmVgPYYOGNy7vz/yygSHICizhGj7v30KySl+3NiLCEAelyygXKOzbnws= ceph-df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:39:02.499 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f5781288700 1 Processor -- start 2024-02-10T06:39:02.499 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f5781288700 1 -- start start 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f5781288700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c108d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f5781288700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c079700 con 0x7f577c106960 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f577affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f577affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53470/0 (socket says 172.21.15.174:53470) 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f577affd700 1 -- 172.21.15.174:0/3129584571 learned_addr learned my addr 172.21.15.174:0/3129584571 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f577affd700 1 -- 172.21.15.174:0/3129584571 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c109290 con 0x7f577c106960 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f577affd700 1 --2- 172.21.15.174:0/3129584571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c108d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5774009e80 tx=0x7f577400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132f33e4b33342e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f5779ffb700 1 -- 172.21.15.174:0/3129584571 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f577400e070 con 0x7f577c106960 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.716+0000 7f5779ffb700 1 -- 172.21.15.174:0/3129584571 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f577400b8c0 con 0x7f577c106960 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5779ffb700 1 -- 172.21.15.174:0/3129584571 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57740043e0 con 0x7f577c106960 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 -- 172.21.15.174:0/3129584571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 msgr2=0x7f577c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 --2- 172.21.15.174:0/3129584571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c108d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5774009e80 tx=0x7f577400b330 comp rx=0 tx=0).stop 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 -- 172.21.15.174:0/3129584571 shutdown_connections 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 --2- 172.21.15.174:0/3129584571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c108d50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 -- 172.21.15.174:0/3129584571 >> 172.21.15.174:0/3129584571 conn(0x7f577c1003b0 msgr2=0x7f577c1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 -- 172.21.15.174:0/3129584571 shutdown_connections 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.717+0000 7f5781288700 1 -- 172.21.15.174:0/3129584571 wait complete. 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f5781288700 1 Processor -- start 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f5781288700 1 -- start start 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f5781288700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f5781288700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c199fb0 con 0x7f577c106960 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f577affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f577affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53482/0 (socket says 172.21.15.174:53482) 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f577affd700 1 -- 172.21.15.174:0/4172164175 learned_addr learned my addr 172.21.15.174:0/4172164175 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f577affd700 1 -- 172.21.15.174:0/4172164175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774009b30 con 0x7f577c106960 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.718+0000 7f577affd700 1 --2- 172.21.15.174:0/4172164175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c199a70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f577400b7b0 tx=0x7f5774004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:02.501 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.719+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f577400e040 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.719+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f577401c8d0 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.719+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5774012790 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.719+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c19a1b0 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.719+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c19a6b0 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.719+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f577401c440 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.720+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c109f50 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.720+0000 7f5773fff700 1 --2- 172.21.15.174:0/4172164175 >> v2:172.21.15.174:6800/1048754011 conn(0x7f5768038e50 0x7f576803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.720+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f577404d260 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.720+0000 7f577a7fc700 1 --2- 172.21.15.174:0/4172164175 >> v2:172.21.15.174:6800/1048754011 conn(0x7f5768038e50 0x7f576803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.720+0000 7f577a7fc700 1 --2- 172.21.15.174:0/4172164175 >> v2:172.21.15.174:6800/1048754011 conn(0x7f5768038e50 0x7f576803b310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5764006fd0 tx=0x7f5764006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.725+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5774011390 con 0x7f577c106960 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.919+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f577c1015b0 con 0x7f5768038e50 2024-02-10T06:39:02.502 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.920+0000 7f5773fff700 1 -- 172.21.15.174:0/4172164175 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f577c1015b0 con 0x7f5768038e50 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 >> v2:172.21.15.174:6800/1048754011 conn(0x7f5768038e50 msgr2=0x7f576803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 --2- 172.21.15.174:0/4172164175 >> v2:172.21.15.174:6800/1048754011 conn(0x7f5768038e50 0x7f576803b310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5764006fd0 tx=0x7f5764006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 msgr2=0x7f577c199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 --2- 172.21.15.174:0/4172164175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c199a70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f577400b7b0 tx=0x7f5774004a20 comp rx=0 tx=0).stop 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 shutdown_connections 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 --2- 172.21.15.174:0/4172164175 >> v2:172.21.15.174:6800/1048754011 conn(0x7f5768038e50 0x7f576803b310 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 --2- 172.21.15.174:0/4172164175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f577c106960 0x7f577c199a70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 >> 172.21.15.174:0/4172164175 conn(0x7f577c1003b0 msgr2=0x7f577c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 shutdown_connections 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:01.922+0000 7f5781288700 1 -- 172.21.15.174:0/4172164175 wait complete. 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:Adding key to root@localhost authorized_keys... 2024-02-10T06:39:02.503 INFO:teuthology.orchestra.run.smithi174.stdout:Adding host smithi174... 2024-02-10T06:39:03.862 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:03 smithi174 ceph-mon[29878]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:04 smithi174 ceph-mon[29878]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "addr": "172.21.15.174", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:05 smithi174 ceph-mon[29878]: Deploying cephadm binary to smithi174 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout Added host 'smithi174' with addr '172.21.15.174' 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.990+0000 7f59fdef4700 1 Processor -- start 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.990+0000 7f59fdef4700 1 -- start start 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59fdef4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f8104b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59fdef4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80796a0 con 0x7f59f8104760 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f8104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f8104b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53490/0 (socket says 172.21.15.174:53490) 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f77fe700 1 -- 172.21.15.174:0/1246596487 learned_addr learned my addr 172.21.15.174:0/1246596487 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:07.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f77fe700 1 -- 172.21.15.174:0/1246596487 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80797e0 con 0x7f59f8104760 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f77fe700 1 --2- 172.21.15.174:0/1246596487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f8104b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098a0 tx=0x7f59e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d0dcf5e772c992 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f67fc700 1 -- 172.21.15.174:0/1246596487 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e8004160 con 0x7f59f8104760 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f67fc700 1 -- 172.21.15.174:0/1246596487 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f59e80042c0 con 0x7f59f8104760 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.991+0000 7f59f67fc700 1 -- 172.21.15.174:0/1246596487 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e80119b0 con 0x7f59f8104760 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 -- 172.21.15.174:0/1246596487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 msgr2=0x7f59f8104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 --2- 172.21.15.174:0/1246596487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f8104b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098a0 tx=0x7f59e8009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 -- 172.21.15.174:0/1246596487 shutdown_connections 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 --2- 172.21.15.174:0/1246596487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f8104b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 -- 172.21.15.174:0/1246596487 >> 172.21.15.174:0/1246596487 conn(0x7f59f8100300 msgr2=0x7f59f8102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 -- 172.21.15.174:0/1246596487 shutdown_connections 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 -- 172.21.15.174:0/1246596487 wait complete. 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.992+0000 7f59fdef4700 1 Processor -- start 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59fdef4700 1 -- start start 2024-02-10T06:39:07.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59fdef4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59fdef4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819c250 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53506/0 (socket says 172.21.15.174:53506) 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f77fe700 1 -- 172.21.15.174:0/3339759192 learned_addr learned my addr 172.21.15.174:0/3339759192 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f77fe700 1 -- 172.21.15.174:0/3339759192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8009550 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f77fe700 1 --2- 172.21.15.174:0/3339759192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f819bd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59e80046c0 tx=0x7f59e8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e80038b0 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7f59e8003a10 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.993+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f819c450 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.994+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e8019690 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.994+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f819c8f0 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.994+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f59e80198c0 con 0x7f59f8104760 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.994+0000 7f59f4ff9700 1 --2- 172.21.15.174:0/3339759192 >> v2:172.21.15.174:6800/1048754011 conn(0x7f59e0038e50 0x7f59e003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:07.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.995+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f59e804d030 con 0x7f59f8104760 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.995+0000 7f59f6ffd700 1 --2- 172.21.15.174:0/3339759192 >> v2:172.21.15.174:6800/1048754011 conn(0x7f59e0038e50 0x7f59e003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.995+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f81959e0 con 0x7f59f8104760 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:02.995+0000 7f59f6ffd700 1 --2- 172.21.15.174:0/3339759192 >> v2:172.21.15.174:6800/1048754011 conn(0x7f59e0038e50 0x7f59e003b310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59ec006fd0 tx=0x7f59ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:03.000+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f59e8023c20 con 0x7f59f8104760 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:03.197+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "addr": "172.21.15.174", "target": ["mon-mgr", ""]}) v1 -- 0x7f59f8062160 con 0x7f59e0038e50 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.383+0000 7f59f4ff9700 1 -- 172.21.15.174:0/3339759192 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f59f8062160 con 0x7f59e0038e50 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 >> v2:172.21.15.174:6800/1048754011 conn(0x7f59e0038e50 msgr2=0x7f59e003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 --2- 172.21.15.174:0/3339759192 >> v2:172.21.15.174:6800/1048754011 conn(0x7f59e0038e50 0x7f59e003b310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59ec006fd0 tx=0x7f59ec006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 msgr2=0x7f59f819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 --2- 172.21.15.174:0/3339759192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f819bd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f59e80046c0 tx=0x7f59e8011a50 comp rx=0 tx=0).stop 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 shutdown_connections 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 --2- 172.21.15.174:0/3339759192 >> v2:172.21.15.174:6800/1048754011 conn(0x7f59e0038e50 0x7f59e003b310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 --2- 172.21.15.174:0/3339759192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104760 0x7f59f819bd10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.385+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 >> 172.21.15.174:0/3339759192 conn(0x7f59f8100300 msgr2=0x7f59f8192b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:07.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.386+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 shutdown_connections 2024-02-10T06:39:07.995 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:07.386+0000 7f59fdef4700 1 -- 172.21.15.174:0/3339759192 wait complete. 2024-02-10T06:39:07.995 INFO:teuthology.orchestra.run.smithi174.stdout:Deploying unmanaged mon service... 2024-02-10T06:39:08.523 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:08 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:08.523 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:08 smithi174 ceph-mon[29878]: Added host smithi174 2024-02-10T06:39:08.523 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:08 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:09.365 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-10T06:39:09.365 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e3fdf700 1 Processor -- start 2024-02-10T06:39:09.365 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e3fdf700 1 -- start start 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e3fdf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e3fdf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc07a180 con 0x7fd9dc07b750 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e1d7b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e1d7b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc079bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46446/0 (socket says 172.21.15.174:46446) 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.522+0000 7fd9e1d7b700 1 -- 172.21.15.174:0/3773432064 learned_addr learned my addr 172.21.15.174:0/3773432064 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e1d7b700 1 -- 172.21.15.174:0/3773432064 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc07a2c0 con 0x7fd9dc07b750 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e1d7b700 1 --2- 172.21.15.174:0/3773432064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc079bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80098a0 tx=0x7fd9d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0ab9df84377578 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e0d79700 1 -- 172.21.15.174:0/3773432064 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d8004160 con 0x7fd9dc07b750 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e0d79700 1 -- 172.21.15.174:0/3773432064 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fd9d80042c0 con 0x7fd9dc07b750 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e0d79700 1 -- 172.21.15.174:0/3773432064 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d80119b0 con 0x7fd9dc07b750 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/3773432064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 msgr2=0x7fd9dc079bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.523+0000 7fd9e3fdf700 1 --2- 172.21.15.174:0/3773432064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc079bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80098a0 tx=0x7fd9d8009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/3773432064 shutdown_connections 2024-02-10T06:39:09.366 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 --2- 172.21.15.174:0/3773432064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc079bb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/3773432064 >> 172.21.15.174:0/3773432064 conn(0x7fd9dc1014d0 msgr2=0x7fd9dc1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/3773432064 shutdown_connections 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/3773432064 wait complete. 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 Processor -- start 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 -- start start 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e3fdf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc19c220 con 0x7fd9dc07b750 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e1d7b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e1d7b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46448/0 (socket says 172.21.15.174:46448) 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.524+0000 7fd9e1d7b700 1 -- 172.21.15.174:0/1806116210 learned_addr learned my addr 172.21.15.174:0/1806116210 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9e1d7b700 1 -- 172.21.15.174:0/1806116210 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8009550 con 0x7fd9dc07b750 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9e1d7b700 1 --2- 172.21.15.174:0/1806116210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc19bce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006af0 tx=0x7fd9d8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d80039e0 con 0x7fd9dc07b750 2024-02-10T06:39:09.367 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fd9d8003b40 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc19c420 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9d8019700 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.525+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc19c8c0 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.526+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fd9d8019930 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.526+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc10c190 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.526+0000 7fd9d2ffd700 1 --2- 172.21.15.174:0/1806116210 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd9c8038ea0 0x7fd9c803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.526+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fd9d804d060 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.526+0000 7fd9e157a700 1 --2- 172.21.15.174:0/1806116210 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd9c8038ea0 0x7fd9c803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.527+0000 7fd9e157a700 1 --2- 172.21.15.174:0/1806116210 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd9c8038ea0 0x7fd9c803b360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.531+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9d8080d20 con 0x7fd9dc07b750 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.738+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd9dc1026d0 con 0x7fd9c8038ea0 2024-02-10T06:39:09.368 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.742+0000 7fd9d2ffd700 1 -- 172.21.15.174:0/1806116210 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd9dc1026d0 con 0x7fd9c8038ea0 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd9c8038ea0 msgr2=0x7fd9c803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 --2- 172.21.15.174:0/1806116210 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd9c8038ea0 0x7fd9c803b360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 msgr2=0x7fd9dc19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 --2- 172.21.15.174:0/1806116210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc19bce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006af0 tx=0x7fd9d8011a50 comp rx=0 tx=0).stop 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 shutdown_connections 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 --2- 172.21.15.174:0/1806116210 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd9c8038ea0 0x7fd9c803b360 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 --2- 172.21.15.174:0/1806116210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc07b750 0x7fd9dc19bce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 >> 172.21.15.174:0/1806116210 conn(0x7fd9dc1014d0 msgr2=0x7fd9dc101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 shutdown_connections 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:08.745+0000 7fd9e3fdf700 1 -- 172.21.15.174:0/1806116210 wait complete. 2024-02-10T06:39:09.369 INFO:teuthology.orchestra.run.smithi174.stdout:Deploying unmanaged mgr service... 2024-02-10T06:39:09.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:09 smithi174 ceph-mon[29878]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:09.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:09 smithi174 ceph-mon[29878]: Saving service mon spec with placement count:5 2024-02-10T06:39:09.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:09 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:11.470 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:11 smithi174 ceph-mon[29878]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:11.470 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:11 smithi174 ceph-mon[29878]: Saving service mgr spec with placement count:2 2024-02-10T06:39:11.470 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:11 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:11.477 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-10T06:39:11.477 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.872+0000 7fc7862d8700 1 Processor -- start 2024-02-10T06:39:11.477 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.872+0000 7fc7862d8700 1 -- start start 2024-02-10T06:39:11.477 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.872+0000 7fc7862d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc780106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:11.477 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.872+0000 7fc7862d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801067c0 con 0x7fc780105e60 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.872+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc780106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc780106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46462/0 (socket says 172.21.15.174:46462) 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77ffff700 1 -- 172.21.15.174:0/4116061106 learned_addr learned my addr 172.21.15.174:0/4116061106 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77ffff700 1 -- 172.21.15.174:0/4116061106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780106900 con 0x7fc780105e60 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77ffff700 1 --2- 172.21.15.174:0/4116061106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc780106280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7740098a0 tx=0x7fc774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605f197585862790 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77effd700 1 -- 172.21.15.174:0/4116061106 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc774004160 con 0x7fc780105e60 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77effd700 1 -- 172.21.15.174:0/4116061106 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fc7740042c0 con 0x7fc780105e60 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc77effd700 1 -- 172.21.15.174:0/4116061106 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7740119b0 con 0x7fc780105e60 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc7862d8700 1 -- 172.21.15.174:0/4116061106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 msgr2=0x7fc780106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.873+0000 7fc7862d8700 1 --2- 172.21.15.174:0/4116061106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc780106280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7740098a0 tx=0x7fc774009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 -- 172.21.15.174:0/4116061106 shutdown_connections 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 --2- 172.21.15.174:0/4116061106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc780106280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 -- 172.21.15.174:0/4116061106 >> 172.21.15.174:0/4116061106 conn(0x7fc7801014d0 msgr2=0x7fc7801038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 -- 172.21.15.174:0/4116061106 shutdown_connections 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 -- 172.21.15.174:0/4116061106 wait complete. 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 Processor -- start 2024-02-10T06:39:11.478 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 -- start start 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc78007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.874+0000 7fc7862d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78007b940 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc78007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc78007b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46478/0 (socket says 172.21.15.174:46478) 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77ffff700 1 -- 172.21.15.174:0/276450371 learned_addr learned my addr 172.21.15.174:0/276450371 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77ffff700 1 -- 172.21.15.174:0/276450371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774009550 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77ffff700 1 --2- 172.21.15.174:0/276450371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc78007b400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7740046c0 tx=0x7fc774011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7740038b0 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1062+0+0 (secure 0 0 0) 0x7fc774003a10 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780079b30 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc774019610 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.875+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780079fd0 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.876+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fc7740198d0 con 0x7fc780105e60 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.876+0000 7fc77d7fa700 1 --2- 172.21.15.174:0/276450371 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc768038e50 0x7fc76803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:11.479 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.876+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc780063350 con 0x7fc780105e60 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.876+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fc77404d160 con 0x7fc780105e60 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.876+0000 7fc77f7fe700 1 --2- 172.21.15.174:0/276450371 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc768038e50 0x7fc76803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.877+0000 7fc77f7fe700 1 --2- 172.21.15.174:0/276450371 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc768038e50 0x7fc76803b310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc770006fd0 tx=0x7fc770006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:09.881+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc774018530 con 0x7fc780105e60 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.099+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 --> v2:172.21.15.174:6800/1048754011 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc7801027b0 con 0x7fc768038e50 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.107+0000 7fc77d7fa700 1 -- 172.21.15.174:0/276450371 <== mgr.14120 v2:172.21.15.174:6800/1048754011 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc7801027b0 con 0x7fc768038e50 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc768038e50 msgr2=0x7fc76803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 --2- 172.21.15.174:0/276450371 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc768038e50 0x7fc76803b310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc770006fd0 tx=0x7fc770006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 msgr2=0x7fc78007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 --2- 172.21.15.174:0/276450371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc78007b400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7740046c0 tx=0x7fc774011a50 comp rx=0 tx=0).stop 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 shutdown_connections 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 --2- 172.21.15.174:0/276450371 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc768038e50 0x7fc76803b310 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:11.480 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 --2- 172.21.15.174:0/276450371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc780105e60 0x7fc78007b400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:11.481 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 >> 172.21.15.174:0/276450371 conn(0x7fc7801014d0 msgr2=0x7fc7801020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:11.481 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 shutdown_connections 2024-02-10T06:39:11.481 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:10.109+0000 7fc7862d8700 1 -- 172.21.15.174:0/276450371 wait complete. 2024-02-10T06:39:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:12 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:12 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:12.758 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.979+0000 7fd8aaa46700 1 Processor -- start 2024-02-10T06:39:12.758 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.979+0000 7fd8aaa46700 1 -- start start 2024-02-10T06:39:12.758 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.979+0000 7fd8aaa46700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a4079bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:12.758 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.979+0000 7fd8aaa46700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a407a1a0 con 0x7fd8a407b770 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.979+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a4079bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a4079bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46490/0 (socket says 172.21.15.174:46490) 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a3fff700 1 -- 172.21.15.174:0/1659267122 learned_addr learned my addr 172.21.15.174:0/1659267122 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a3fff700 1 -- 172.21.15.174:0/1659267122 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a407a2e0 con 0x7fd8a407b770 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a3fff700 1 --2- 172.21.15.174:0/1659267122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a4079bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8980098a0 tx=0x7fd898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a450e9ece72a818 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a2ffd700 1 -- 172.21.15.174:0/1659267122 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd898004160 con 0x7fd8a407b770 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a2ffd700 1 -- 172.21.15.174:0/1659267122 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd8980042c0 con 0x7fd8a407b770 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.980+0000 7fd8a2ffd700 1 -- 172.21.15.174:0/1659267122 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd898011960 con 0x7fd8a407b770 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 -- 172.21.15.174:0/1659267122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 msgr2=0x7fd8a4079bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 --2- 172.21.15.174:0/1659267122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a4079bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8980098a0 tx=0x7fd898009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 -- 172.21.15.174:0/1659267122 shutdown_connections 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 --2- 172.21.15.174:0/1659267122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a4079bd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 -- 172.21.15.174:0/1659267122 >> 172.21.15.174:0/1659267122 conn(0x7fd8a4101420 msgr2=0x7fd8a41038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 -- 172.21.15.174:0/1659267122 shutdown_connections 2024-02-10T06:39:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.981+0000 7fd8aaa46700 1 -- 172.21.15.174:0/1659267122 wait complete. 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8aaa46700 1 Processor -- start 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8aaa46700 1 -- start start 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8aaa46700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8aaa46700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419c2f0 con 0x7fd8a407b770 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46506/0 (socket says 172.21.15.174:46506) 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.982+0000 7fd8a3fff700 1 -- 172.21.15.174:0/3801867015 learned_addr learned my addr 172.21.15.174:0/3801867015 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8a3fff700 1 -- 172.21.15.174:0/3801867015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898009550 con 0x7fd8a407b770 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8a3fff700 1 --2- 172.21.15.174:0/3801867015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a419bdb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd898009fb0 tx=0x7fd8980119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd898003970 con 0x7fd8a407b770 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a419c4f0 con 0x7fd8a407b770 2024-02-10T06:39:12.760 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd898003ad0 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd898019630 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.983+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a419c990 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.984+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fd8980198f0 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.984+0000 7fd8a17fa700 1 --2- 172.21.15.174:0/3801867015 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd88c038e50 0x7fd88c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.984+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a410c2a0 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.984+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fd89804d080 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.984+0000 7fd8a37fe700 1 --2- 172.21.15.174:0/3801867015 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd88c038e50 0x7fd88c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.985+0000 7fd8a37fe700 1 --2- 172.21.15.174:0/3801867015 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd88c038e50 0x7fd88c03b310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd894006fd0 tx=0x7fd894006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:11.989+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd89802b340 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.185+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd8a419f370 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.190+0000 7fd8a17fa700 1 -- 172.21.15.174:0/3801867015 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd898018be0 con 0x7fd8a407b770 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd88c038e50 msgr2=0x7fd88c03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 --2- 172.21.15.174:0/3801867015 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd88c038e50 0x7fd88c03b310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd894006fd0 tx=0x7fd894006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 msgr2=0x7fd8a419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:12.761 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 --2- 172.21.15.174:0/3801867015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a419bdb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd898009fb0 tx=0x7fd8980119e0 comp rx=0 tx=0).stop 2024-02-10T06:39:12.762 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 shutdown_connections 2024-02-10T06:39:12.762 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 --2- 172.21.15.174:0/3801867015 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd88c038e50 0x7fd88c03b310 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:12.762 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 --2- 172.21.15.174:0/3801867015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a407b770 0x7fd8a419bdb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:12.762 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 >> 172.21.15.174:0/3801867015 conn(0x7fd8a4101420 msgr2=0x7fd8a4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:12.762 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 shutdown_connections 2024-02-10T06:39:12.762 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:12.192+0000 7fd8aaa46700 1 -- 172.21.15.174:0/3801867015 wait complete. 2024-02-10T06:39:13.444 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3801867015' entity='client.admin' 2024-02-10T06:39:14.671 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/854702971' entity='client.admin' 2024-02-10T06:39:14.671 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:14 smithi174 ceph-mon[29878]: from='mgr.14120 172.21.15.174:0/2723512793' entity='mgr.y' 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f680d526700 1 Processor -- start 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f680d526700 1 -- start start 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f680d526700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f68081051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f680d526700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081057a0 con 0x7f6808104db0 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f68081051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f68081051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46518/0 (socket says 172.21.15.174:46518) 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f6806ffd700 1 -- 172.21.15.174:0/1663946849 learned_addr learned my addr 172.21.15.174:0/1663946849 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f6806ffd700 1 -- 172.21.15.174:0/1663946849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808105fb0 con 0x7f6808104db0 2024-02-10T06:39:14.678 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.215+0000 7f6806ffd700 1 --2- 172.21.15.174:0/1663946849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f68081051d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098a0 tx=0x7f67f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4a3ff98235e783 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f6805ffb700 1 -- 172.21.15.174:0/1663946849 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67f8004160 con 0x7f6808104db0 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f6805ffb700 1 -- 172.21.15.174:0/1663946849 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f67f80042c0 con 0x7f6808104db0 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f6805ffb700 1 -- 172.21.15.174:0/1663946849 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67f8011960 con 0x7f6808104db0 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 -- 172.21.15.174:0/1663946849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 msgr2=0x7f68081051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 --2- 172.21.15.174:0/1663946849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f68081051d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098a0 tx=0x7f67f8009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 -- 172.21.15.174:0/1663946849 shutdown_connections 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 --2- 172.21.15.174:0/1663946849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f68081051d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 -- 172.21.15.174:0/1663946849 >> 172.21.15.174:0/1663946849 conn(0x7f68081003b0 msgr2=0x7f68081027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 -- 172.21.15.174:0/1663946849 shutdown_connections 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.216+0000 7f680d526700 1 -- 172.21.15.174:0/1663946849 wait complete. 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f680d526700 1 Processor -- start 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f680d526700 1 -- start start 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f680d526700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f6808199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:14.679 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f680d526700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680819a160 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f6808199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f6806ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f6808199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46534/0 (socket says 172.21.15.174:46534) 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.217+0000 7f6806ffd700 1 -- 172.21.15.174:0/854702971 learned_addr learned my addr 172.21.15.174:0/854702971 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f6806ffd700 1 -- 172.21.15.174:0/854702971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009550 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f6806ffd700 1 --2- 172.21.15.174:0/854702971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f6808199c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006af0 tx=0x7f67f8011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67f80036a0 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f67f801a430 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680819a360 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67f80193c0 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.218+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680819a800 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.219+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7f67f8019640 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.219+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808193d30 con 0x7f6808104db0 2024-02-10T06:39:14.680 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.219+0000 7f67effff700 1 --2- 172.21.15.174:0/854702971 >> v2:172.21.15.174:6800/1048754011 conn(0x7f67f0038ea0 0x7f67f003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.219+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7f67f804cb30 con 0x7f6808104db0 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.219+0000 7f68067fc700 1 --2- 172.21.15.174:0/854702971 >> v2:172.21.15.174:6800/1048754011 conn(0x7f67f0038ea0 0x7f67f003b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.219+0000 7f68067fc700 1 --2- 172.21.15.174:0/854702971 >> v2:172.21.15.174:6800/1048754011 conn(0x7f67f0038ea0 0x7f67f003b360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.224+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67f8019e10 con 0x7f6808104db0 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.443+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f68081091e0 con 0x7f6808104db0 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.446+0000 7f67effff700 1 -- 172.21.15.174:0/854702971 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f67f8010600 con 0x7f6808104db0 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 >> v2:172.21.15.174:6800/1048754011 conn(0x7f67f0038ea0 msgr2=0x7f67f003b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 --2- 172.21.15.174:0/854702971 >> v2:172.21.15.174:6800/1048754011 conn(0x7f67f0038ea0 0x7f67f003b360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 msgr2=0x7f6808199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 --2- 172.21.15.174:0/854702971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f6808199c20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006af0 tx=0x7f67f8011eb0 comp rx=0 tx=0).stop 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 shutdown_connections 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 --2- 172.21.15.174:0/854702971 >> v2:172.21.15.174:6800/1048754011 conn(0x7f67f0038ea0 0x7f67f003b360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 --2- 172.21.15.174:0/854702971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6808104db0 0x7f6808199c20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:14.681 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 >> 172.21.15.174:0/854702971 conn(0x7f68081003b0 msgr2=0x7f68081012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:14.682 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 shutdown_connections 2024-02-10T06:39:14.682 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:13.449+0000 7f680d526700 1 -- 172.21.15.174:0/854702971 wait complete. 2024-02-10T06:39:14.682 INFO:teuthology.orchestra.run.smithi174.stdout:Enabling the dashboard module... 2024-02-10T06:39:15.736 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2988839079' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-10T06:39:15.736 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:15 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ignoring --setuser ceph since I am not root 2024-02-10T06:39:15.736 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:15 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ignoring --setgroup ceph since I am not root 2024-02-10T06:39:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc12171d700 1 Processor -- start 2024-02-10T06:39:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc12171d700 1 -- start start 2024-02-10T06:39:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc12171d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c1067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc12171d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c106d70 con 0x7fc11c106380 2024-02-10T06:39:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc11affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c1067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc11affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c1067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46540/0 (socket says 172.21.15.174:46540) 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.123+0000 7fc11affd700 1 -- 172.21.15.174:0/2549341275 learned_addr learned my addr 172.21.15.174:0/2549341275 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.124+0000 7fc11affd700 1 -- 172.21.15.174:0/2549341275 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c106eb0 con 0x7fc11c106380 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.124+0000 7fc11affd700 1 --2- 172.21.15.174:0/2549341275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c1067a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098a0 tx=0x7fc10c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bb8a372bf27bfa server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.124+0000 7fc119ffb700 1 -- 172.21.15.174:0/2549341275 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c004160 con 0x7fc11c106380 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.124+0000 7fc119ffb700 1 -- 172.21.15.174:0/2549341275 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc10c0042c0 con 0x7fc11c106380 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.124+0000 7fc119ffb700 1 -- 172.21.15.174:0/2549341275 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c011960 con 0x7fc11c106380 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 -- 172.21.15.174:0/2549341275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 msgr2=0x7fc11c1067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 --2- 172.21.15.174:0/2549341275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c1067a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098a0 tx=0x7fc10c009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 -- 172.21.15.174:0/2549341275 shutdown_connections 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 --2- 172.21.15.174:0/2549341275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c1067a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 -- 172.21.15.174:0/2549341275 >> 172.21.15.174:0/2549341275 conn(0x7fc11c0ffa40 msgr2=0x7fc11c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 -- 172.21.15.174:0/2549341275 shutdown_connections 2024-02-10T06:39:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 -- 172.21.15.174:0/2549341275 wait complete. 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.125+0000 7fc12171d700 1 Processor -- start 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc12171d700 1 -- start start 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc12171d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc12171d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19c220 con 0x7fc11c106380 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc11affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc11affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:46554/0 (socket says 172.21.15.174:46554) 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc11affd700 1 -- 172.21.15.174:0/2988839079 learned_addr learned my addr 172.21.15.174:0/2988839079 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc11affd700 1 -- 172.21.15.174:0/2988839079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009550 con 0x7fc11c106380 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc11affd700 1 --2- 172.21.15.174:0/2988839079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c19bce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006af0 tx=0x7fc10c0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c0038b0 con 0x7fc11c106380 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc10c003a10 con 0x7fc11c106380 2024-02-10T06:39:16.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c19c420 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c019610 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.126+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c19c8c0 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.127+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 8) v1 ==== 45897+0+0 (secure 0 0 0) 0x7fc10c019840 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.127+0000 7fc103fff700 1 --2- 172.21.15.174:0/2988839079 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc104038ea0 0x7fc10403b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.128+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fc10c01a430 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.128+0000 7fc11a7fc700 1 --2- 172.21.15.174:0/2988839079 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc104038ea0 0x7fc10403b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.128+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c104b30 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.128+0000 7fc11a7fc700 1 --2- 172.21.15.174:0/2988839079 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc104038ea0 0x7fc10403b360 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc110006fd0 tx=0x7fc110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.133+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc10c019af0 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.387+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc11c195450 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.450+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc10c018be0 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.452+0000 7fc103fff700 1 -- 172.21.15.174:0/2988839079 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mgrmap(e 9) v1 ==== 45910+0+0 (secure 0 0 0) 0x7fc10c019cd0 con 0x7fc11c106380 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc104038ea0 msgr2=0x7fc10403b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 --2- 172.21.15.174:0/2988839079 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc104038ea0 0x7fc10403b360 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc110006fd0 tx=0x7fc110006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:16.030 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 msgr2=0x7fc11c19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 --2- 172.21.15.174:0/2988839079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c19bce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006af0 tx=0x7fc10c0119c0 comp rx=0 tx=0).stop 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 shutdown_connections 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 --2- 172.21.15.174:0/2988839079 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc104038ea0 0x7fc10403b360 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 --2- 172.21.15.174:0/2988839079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11c106380 0x7fc11c19bce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 >> 172.21.15.174:0/2988839079 conn(0x7fc11c0ffa40 msgr2=0x7fc11c192510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 shutdown_connections 2024-02-10T06:39:16.031 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:15.454+0000 7fc12171d700 1 -- 172.21.15.174:0/2988839079 wait complete. 2024-02-10T06:39:16.274 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:16 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:16.019+0000 7f2bc95da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:39:16.274 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:16 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:16.117+0000 7f2bc95da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:39:16.696 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2988839079' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-10T06:39:16.696 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:16 smithi174 ceph-mon[29878]: mgrmap e9: y(active, since 19s) 2024-02-10T06:39:16.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:16 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:16.787+0000 7f2bc95da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:39:16.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:16 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:16.881+0000 7f2bc95da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:39:17.409 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.025+0000 7f2bc95da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:39:17.409 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.121+0000 7f2bc95da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:39:17.409 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.205+0000 7f2bc95da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.489+0000 7fd1bd20a700 1 Processor -- start 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.489+0000 7fd1bd20a700 1 -- start start 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.489+0000 7fd1bd20a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.489+0000 7fd1bd20a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b81067a0 con 0x7fd1b8105e40 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.489+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48456/0 (socket says 172.21.15.174:48456) 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b6d9d700 1 -- 172.21.15.174:0/345301318 learned_addr learned my addr 172.21.15.174:0/345301318 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b6d9d700 1 -- 172.21.15.174:0/345301318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b81068e0 con 0x7fd1b8105e40 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b6d9d700 1 --2- 172.21.15.174:0/345301318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8106260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0098a0 tx=0x7fd1ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2e01c81c398dbf server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b5d9b700 1 -- 172.21.15.174:0/345301318 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac004160 con 0x7fd1b8105e40 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b5d9b700 1 -- 172.21.15.174:0/345301318 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd1ac0042c0 con 0x7fd1b8105e40 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.490+0000 7fd1b5d9b700 1 -- 172.21.15.174:0/345301318 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac011960 con 0x7fd1b8105e40 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 -- 172.21.15.174:0/345301318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 msgr2=0x7fd1b8106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 --2- 172.21.15.174:0/345301318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8106260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0098a0 tx=0x7fd1ac009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 -- 172.21.15.174:0/345301318 shutdown_connections 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 --2- 172.21.15.174:0/345301318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8106260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 -- 172.21.15.174:0/345301318 >> 172.21.15.174:0/345301318 conn(0x7fd1b81013a0 msgr2=0x7fd1b8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 -- 172.21.15.174:0/345301318 shutdown_connections 2024-02-10T06:39:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 -- 172.21.15.174:0/345301318 wait complete. 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.491+0000 7fd1bd20a700 1 Processor -- start 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1bd20a700 1 -- start start 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1bd20a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1bd20a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b819a080 con 0x7fd1b8105e40 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1b6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48460/0 (socket says 172.21.15.174:48460) 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1b6d9d700 1 -- 172.21.15.174:0/148256944 learned_addr learned my addr 172.21.15.174:0/148256944 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1b6d9d700 1 -- 172.21.15.174:0/148256944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac009550 con 0x7fd1b8105e40 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.492+0000 7fd1b6d9d700 1 --2- 172.21.15.174:0/148256944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8199b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac003f60 tx=0x7fd1ac0119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.493+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac003970 con 0x7fd1b8105e40 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.493+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b819a280 con 0x7fd1b8105e40 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.493+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd1ac003ad0 con 0x7fd1b8105e40 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.493+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac019750 con 0x7fd1b8105e40 2024-02-10T06:39:17.420 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.493+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b819a720 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.494+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b8063350 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.494+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 9) v1 ==== 45910+0+0 (secure 0 0 0) 0x7fd1ac0199d0 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.494+0000 7fd19ffff700 1 --2- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 0x7fd1a003b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.494+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fd1ac04cfb0 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.494+0000 7fd1b659c700 1 -- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 msgr2=0x7fd1a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.494+0000 7fd1b659c700 1 --2- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 0x7fd1a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.499+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1ac018760 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.695+0000 7fd1b659c700 1 -- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 msgr2=0x7fd1a003b360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.695+0000 7fd1b659c700 1 --2- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 0x7fd1a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.746+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd1b819d000 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.746+0000 7fd19ffff700 1 -- 172.21.15.174:0/148256944 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fd1ac018350 con 0x7fd1b8105e40 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 msgr2=0x7fd1a003b360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 --2- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 0x7fd1a003b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 msgr2=0x7fd1b8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 --2- 172.21.15.174:0/148256944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8199b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac003f60 tx=0x7fd1ac0119e0 comp rx=0 tx=0).stop 2024-02-10T06:39:17.421 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 shutdown_connections 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 --2- 172.21.15.174:0/148256944 >> v2:172.21.15.174:6800/1048754011 conn(0x7fd1a0038ea0 0x7fd1a003b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 --2- 172.21.15.174:0/148256944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1b8105e40 0x7fd1b8199b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 >> 172.21.15.174:0/148256944 conn(0x7fd1b81013a0 msgr2=0x7fd1b8101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 shutdown_connections 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:16.749+0000 7fd1bd20a700 1 -- 172.21.15.174:0/148256944 wait complete. 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for the mgr to restart... 2024-02-10T06:39:17.422 INFO:teuthology.orchestra.run.smithi174.stdout:Waiting for mgr epoch 9... 2024-02-10T06:39:17.662 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/148256944' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T06:39:17.663 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.475+0000 7f2bc95da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:39:17.663 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.587+0000 7f2bc95da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:39:17.966 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.684+0000 7f2bc95da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:39:17.966 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.780+0000 7f2bc95da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:39:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:17.965+0000 7f2bc95da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:39:19.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:18.923+0000 7f2bc95da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:39:19.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:19 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:19.053+0000 7f2bc95da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:39:19.671 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:19 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:19.387+0000 7f2bc95da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:39:19.672 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:19 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:19.476+0000 7f2bc95da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:39:19.672 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:19 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:19.670+0000 7f2bc95da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:39:19.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:19 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:19.854+0000 7f2bc95da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:39:20.397 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:20 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:20.138+0000 7f2bc95da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:39:20.397 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:20 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:20.395+0000 7f2bc95da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:39:20.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:20 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:20.630+0000 7f2bc95da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:39:21.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:21 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:21.425+0000 7f2bc95da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:39:22.311 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:21 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:21.994+0000 7f2bc95da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:39:22.311 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:22.092+0000 7f2bc95da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:39:22.311 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:39:22.309+0000 7f2bc95da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: Active manager daemon y restarted 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: Activating manager daemon y 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: osdmap e3: 0 total, 0 up, 0 in 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: mgrmap e10: y(active, starting, since 0.00309969s) 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:39:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:39:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:39:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: Manager daemon y is now available 2024-02-10T06:39:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:23.633 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:39:23.633 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:39:23.633 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:23 smithi174 ceph-mon[29878]: mgrmap e11: y(active, since 1.00556s) 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout { 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout } 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.911+0000 7fc70ddf6700 1 Processor -- start 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.911+0000 7fc70ddf6700 1 -- start start 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.911+0000 7fc70ddf6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708104510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.911+0000 7fc70ddf6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708104ae0 con 0x7fc7081040f0 2024-02-10T06:39:24.025 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7077fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708104510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7077fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708104510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48464/0 (socket says 172.21.15.174:48464) 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7077fe700 1 -- 172.21.15.174:0/1765451765 learned_addr learned my addr 172.21.15.174:0/1765451765 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7077fe700 1 -- 172.21.15.174:0/1765451765 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708107080 con 0x7fc7081040f0 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7077fe700 1 --2- 172.21.15.174:0/1765451765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708104510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098a0 tx=0x7fc6fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f986b0270d16b25c server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7067fc700 1 -- 172.21.15.174:0/1765451765 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc004160 con 0x7fc7081040f0 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7067fc700 1 -- 172.21.15.174:0/1765451765 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc6fc0042c0 con 0x7fc7081040f0 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.912+0000 7fc7067fc700 1 -- 172.21.15.174:0/1765451765 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc011960 con 0x7fc7081040f0 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 -- 172.21.15.174:0/1765451765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 msgr2=0x7fc708104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/1765451765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708104510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098a0 tx=0x7fc6fc009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 -- 172.21.15.174:0/1765451765 shutdown_connections 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/1765451765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708104510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 -- 172.21.15.174:0/1765451765 >> 172.21.15.174:0/1765451765 conn(0x7fc7080ffa40 msgr2=0x7fc708101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 -- 172.21.15.174:0/1765451765 shutdown_connections 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.913+0000 7fc70ddf6700 1 -- 172.21.15.174:0/1765451765 wait complete. 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc70ddf6700 1 Processor -- start 2024-02-10T06:39:24.026 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc70ddf6700 1 -- start start 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc70ddf6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc70ddf6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70819a030 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc7077fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc7077fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48478/0 (socket says 172.21.15.174:48478) 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc7077fe700 1 -- 172.21.15.174:0/384363401 learned_addr learned my addr 172.21.15.174:0/384363401 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc7077fe700 1 -- 172.21.15.174:0/384363401 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc009550 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.914+0000 7fc7077fe700 1 --2- 172.21.15.174:0/384363401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708199af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.915+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc0036a0 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.915+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc6fc024440 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.915+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70819a230 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.915+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc01a430 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.915+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70819a6d0 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.916+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 9) v1 ==== 45910+0+0 (secure 0 0 0) 0x7fc6fc01a6f0 con 0x7fc7081040f0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.916+0000 7fc704ff9700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.916+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 --> v2:172.21.15.174:6800/1048754011 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6f003ba80 con 0x7fc6f0038ef0 2024-02-10T06:39:24.027 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.916+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (secure 0 0 0) 0x7fc6fc04c920 con 0x7fc7081040f0 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.916+0000 7fc706ffd700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 msgr2=0x7fc6f003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:17.916+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:18.117+0000 7fc706ffd700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 msgr2=0x7fc6f003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:18.117+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:18.518+0000 7fc706ffd700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 msgr2=0x7fc6f003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:18.518+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:19.319+0000 7fc706ffd700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 msgr2=0x7fc6f003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:19.319+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:20.919+0000 7fc706ffd700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 msgr2=0x7fc6f003b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/1048754011 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:20.919+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:22.324+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7fc6fc02d910 con 0x7fc7081040f0 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:22.324+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 msgr2=0x7fc6f003b3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:22.324+0000 7fc704ff9700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.325+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mgrmap(e 11) v1 ==== 45804+0+0 (secure 0 0 0) 0x7fc6fc010d90 con 0x7fc7081040f0 2024-02-10T06:39:24.028 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.325+0000 7fc704ff9700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc6f003c900 0x7fc6f003ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.325+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 --> v2:172.21.15.174:6800/2183136494 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6f003ba80 con 0x7fc6f003c900 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.326+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc6f003c900 0x7fc6f003ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.327+0000 7fc706ffd700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc6f003c900 0x7fc6f003ecf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8003a10 tx=0x7fc6f80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.327+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc6f003ba80 con 0x7fc6f003c900 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 --> v2:172.21.15.174:6800/2183136494 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc70810a2b0 con 0x7fc6f003c900 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc704ff9700 1 -- 172.21.15.174:0/384363401 <== mgr.14152 v2:172.21.15.174:6800/2183136494 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc70810a2b0 con 0x7fc6f003c900 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc6f003c900 msgr2=0x7fc6f003ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc6f003c900 0x7fc6f003ecf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8003a10 tx=0x7fc6f80092b0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 msgr2=0x7fc708199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/384363401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708199af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc011eb0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 shutdown_connections 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc6f003c900 0x7fc6f003ecf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/384363401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7081040f0 0x7fc708199af0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.334+0000 7fc70ddf6700 1 --2- 172.21.15.174:0/384363401 >> v2:172.21.15.174:6800/1048754011 conn(0x7fc6f0038ef0 0x7fc6f003b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.335+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 >> 172.21.15.174:0/384363401 conn(0x7fc7080ffa40 msgr2=0x7fc7081004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.335+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 shutdown_connections 2024-02-10T06:39:24.029 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:23.335+0000 7fc70ddf6700 1 -- 172.21.15.174:0/384363401 wait complete. 2024-02-10T06:39:24.030 INFO:teuthology.orchestra.run.smithi174.stdout:mgr epoch 9 is available 2024-02-10T06:39:24.030 INFO:teuthology.orchestra.run.smithi174.stdout:Generating a dashboard self-signed certificate... 2024-02-10T06:39:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:24 smithi174 ceph-mon[29878]: [10/Feb/2024:06:39:23] ENGINE Bus STARTING 2024-02-10T06:39:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:24 smithi174 ceph-mon[29878]: [10/Feb/2024:06:39:23] ENGINE Serving on https://172.21.15.174:7150 2024-02-10T06:39:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:24 smithi174 ceph-mon[29878]: [10/Feb/2024:06:39:23] ENGINE Serving on http://172.21.15.174:8765 2024-02-10T06:39:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:24 smithi174 ceph-mon[29878]: [10/Feb/2024:06:39:23] ENGINE Bus STARTED 2024-02-10T06:39:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:24 smithi174 ceph-mon[29878]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T06:39:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:24 smithi174 ceph-mon[29878]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T06:39:25.424 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:25 smithi174 ceph-mon[29878]: mgrmap e12: y(active, since 2s) 2024-02-10T06:39:25.424 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.941+0000 7f918c70b700 1 Processor -- start 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.941+0000 7f918c70b700 1 -- start start 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918c70b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184108d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918c70b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184078dd0 con 0x7f9184106950 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918a4a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918a4a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48554/0 (socket says 172.21.15.174:48554) 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918a4a7700 1 -- 172.21.15.174:0/2839581405 learned_addr learned my addr 172.21.15.174:0/2839581405 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:25.990 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918a4a7700 1 -- 172.21.15.174:0/2839581405 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184078f10 con 0x7f9184106950 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f918a4a7700 1 --2- 172.21.15.174:0/2839581405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184108d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91740098a0 tx=0x7f9174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed32d87a157cc6b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.942+0000 7f91894a5700 1 -- 172.21.15.174:0/2839581405 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9174004160 con 0x7f9184106950 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f91894a5700 1 -- 172.21.15.174:0/2839581405 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f91740042c0 con 0x7f9184106950 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f91894a5700 1 -- 172.21.15.174:0/2839581405 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9174011960 con 0x7f9184106950 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 -- 172.21.15.174:0/2839581405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 msgr2=0x7f9184108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 --2- 172.21.15.174:0/2839581405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184108d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91740098a0 tx=0x7f9174009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 -- 172.21.15.174:0/2839581405 shutdown_connections 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 --2- 172.21.15.174:0/2839581405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184108d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 -- 172.21.15.174:0/2839581405 >> 172.21.15.174:0/2839581405 conn(0x7f91841002c0 msgr2=0x7f91841026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 -- 172.21.15.174:0/2839581405 shutdown_connections 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.943+0000 7f918c70b700 1 -- 172.21.15.174:0/2839581405 wait complete. 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918c70b700 1 Processor -- start 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918c70b700 1 -- start start 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918c70b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918c70b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184199fe0 con 0x7f9184106950 2024-02-10T06:39:25.991 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918a4a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918a4a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48556/0 (socket says 172.21.15.174:48556) 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918a4a7700 1 -- 172.21.15.174:0/3963343484 learned_addr learned my addr 172.21.15.174:0/3963343484 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.944+0000 7f918a4a7700 1 -- 172.21.15.174:0/3963343484 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174009550 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.945+0000 7f918a4a7700 1 --2- 172.21.15.174:0/3963343484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184199aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9174006af0 tx=0x7f91740119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.945+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91740038b0 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.945+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918419a1e0 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.945+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9174003a10 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.945+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9174019610 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.945+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918419a680 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.946+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 12) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f917401a430 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.946+0000 7f917b7fe700 1 --2- 172.21.15.174:0/3963343484 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9170038ef0 0x7f917003b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.946+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9184109ff0 con 0x7f9184106950 2024-02-10T06:39:25.992 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.946+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f917404d220 con 0x7f9184106950 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.946+0000 7f9189ca6700 1 --2- 172.21.15.174:0/3963343484 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9170038ef0 0x7f917003b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.947+0000 7f9189ca6700 1 --2- 172.21.15.174:0/3963343484 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9170038ef0 0x7f917003b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9180006fd0 tx=0x7f9180006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:24.951+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f917402d440 con 0x7f9184106950 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.157+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9184101560 con 0x7f9170038ef0 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.403+0000 7f917b7fe700 1 -- 172.21.15.174:0/3963343484 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9184101560 con 0x7f9170038ef0 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9170038ef0 msgr2=0x7f917003b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 --2- 172.21.15.174:0/3963343484 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9170038ef0 0x7f917003b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9180006fd0 tx=0x7f9180006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 msgr2=0x7f9184199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 --2- 172.21.15.174:0/3963343484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184199aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9174006af0 tx=0x7f91740119c0 comp rx=0 tx=0).stop 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 shutdown_connections 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 --2- 172.21.15.174:0/3963343484 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9170038ef0 0x7f917003b3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 --2- 172.21.15.174:0/3963343484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9184106950 0x7f9184199aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 >> 172.21.15.174:0/3963343484 conn(0x7f91841002c0 msgr2=0x7f9184100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:25.993 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 shutdown_connections 2024-02-10T06:39:25.994 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:25.407+0000 7f918c70b700 1 -- 172.21.15.174:0/3963343484 wait complete. 2024-02-10T06:39:25.994 INFO:teuthology.orchestra.run.smithi174.stdout:Creating initial admin user... 2024-02-10T06:39:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:26 smithi174 ceph-mon[29878]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:27.925 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1zt1rV1niu1M7tho5223TefkROAEvD/.SexZeE3BWJpWt.JfEg09C", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707547166, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.472+0000 7fa9a3c8a700 1 Processor -- start 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.472+0000 7fa9a3c8a700 1 -- start start 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a3c8a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c104500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a3c8a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c104ad0 con 0x7fa99c1040e0 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a1a26700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a1a26700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c104500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56496/0 (socket says 172.21.15.174:56496) 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a1a26700 1 -- 172.21.15.174:0/493177855 learned_addr learned my addr 172.21.15.174:0/493177855 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a1a26700 1 -- 172.21.15.174:0/493177855 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c104c10 con 0x7fa99c1040e0 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.473+0000 7fa9a1a26700 1 --2- 172.21.15.174:0/493177855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c104500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098a0 tx=0x7fa98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0336245ba5252c server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.474+0000 7fa9a0a24700 1 -- 172.21.15.174:0/493177855 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa98c004160 con 0x7fa99c1040e0 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.474+0000 7fa9a0a24700 1 -- 172.21.15.174:0/493177855 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa98c0042c0 con 0x7fa99c1040e0 2024-02-10T06:39:27.926 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.474+0000 7fa9a0a24700 1 -- 172.21.15.174:0/493177855 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa98c011960 con 0x7fa99c1040e0 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.474+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/493177855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 msgr2=0x7fa99c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.474+0000 7fa9a3c8a700 1 --2- 172.21.15.174:0/493177855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c104500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098a0 tx=0x7fa98c009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/493177855 shutdown_connections 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 --2- 172.21.15.174:0/493177855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c104500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/493177855 >> 172.21.15.174:0/493177855 conn(0x7fa99c0ffa40 msgr2=0x7fa99c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/493177855 shutdown_connections 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/493177855 wait complete. 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 Processor -- start 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.475+0000 7fa9a3c8a700 1 -- start start 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a3c8a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c1069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a3c8a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c106f20 con 0x7fa99c1040e0 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a1a26700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c1069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a1a26700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c1069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56506/0 (socket says 172.21.15.174:56506) 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a1a26700 1 -- 172.21.15.174:0/2729720287 learned_addr learned my addr 172.21.15.174:0/2729720287 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:27.927 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a1a26700 1 -- 172.21.15.174:0/2729720287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009550 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a1a26700 1 --2- 172.21.15.174:0/2729720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c1069e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa98c0036a0 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa98c024440 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c105110 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.476+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa98c01a430 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.478+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 12) v1 ==== 45910+0+0 (secure 0 0 0) 0x7fa98c01a6f0 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.478+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c105610 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.478+0000 7fa992ffd700 1 --2- 172.21.15.174:0/2729720287 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa988038ef0 0x7fa98803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.478+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fa98c024820 con 0x7fa99c1040e0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.478+0000 7fa9a1225700 1 --2- 172.21.15.174:0/2729720287 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa988038ef0 0x7fa98803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.478+0000 7fa9a1225700 1 --2- 172.21.15.174:0/2729720287 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa988038ef0 0x7fa98803b3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa998006fd0 tx=0x7fa998006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:27.928 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.479+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c063350 con 0x7fa99c1040e0 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.488+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa98c053060 con 0x7fa99c1040e0 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.720+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa99c063170 con 0x7fa988038ef0 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.948+0000 7fa992ffd700 1 -- 172.21.15.174:0/2729720287 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fa99c063170 con 0x7fa988038ef0 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa988038ef0 msgr2=0x7fa98803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 --2- 172.21.15.174:0/2729720287 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa988038ef0 0x7fa98803b3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa998006fd0 tx=0x7fa998006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 msgr2=0x7fa99c1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 --2- 172.21.15.174:0/2729720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c1069e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c011eb0 comp rx=0 tx=0).stop 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 shutdown_connections 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 --2- 172.21.15.174:0/2729720287 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa988038ef0 0x7fa98803b3b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 --2- 172.21.15.174:0/2729720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa99c1040e0 0x7fa99c1069e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 >> 172.21.15.174:0/2729720287 conn(0x7fa99c0ffa40 msgr2=0x7fa99c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 shutdown_connections 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:26.951+0000 7fa9a3c8a700 1 -- 172.21.15.174:0/2729720287 wait complete. 2024-02-10T06:39:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:Fetching dashboard port number... 2024-02-10T06:39:28.169 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:27 smithi174 ceph-mon[29878]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:28.169 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:27 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:29.226 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3912708113' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stdout 8443 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.718+0000 7f18f2f80700 1 Processor -- start 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.718+0000 7f18f2f80700 1 -- start start 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.718+0000 7f18f2f80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec1044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.718+0000 7f18f2f80700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec104ab0 con 0x7f18ec1040c0 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18f0d1c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec1044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:29.466 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18f0d1c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec1044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56508/0 (socket says 172.21.15.174:56508) 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18f0d1c700 1 -- 172.21.15.174:0/935948277 learned_addr learned my addr 172.21.15.174:0/935948277 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18f0d1c700 1 -- 172.21.15.174:0/935948277 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec104bf0 con 0x7f18ec1040c0 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18f0d1c700 1 --2- 172.21.15.174:0/935948277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec1044e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18e00098a0 tx=0x7f18e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571281a823b3d279 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18eb7fe700 1 -- 172.21.15.174:0/935948277 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18e0004160 con 0x7f18ec1040c0 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18eb7fe700 1 -- 172.21.15.174:0/935948277 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f18e00042c0 con 0x7f18ec1040c0 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.719+0000 7f18eb7fe700 1 -- 172.21.15.174:0/935948277 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18e0011960 con 0x7f18ec1040c0 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 -- 172.21.15.174:0/935948277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 msgr2=0x7f18ec1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 --2- 172.21.15.174:0/935948277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec1044e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18e00098a0 tx=0x7f18e0009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 -- 172.21.15.174:0/935948277 shutdown_connections 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 --2- 172.21.15.174:0/935948277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec1044e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 -- 172.21.15.174:0/935948277 >> 172.21.15.174:0/935948277 conn(0x7f18ec0ffa40 msgr2=0x7f18ec101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 -- 172.21.15.174:0/935948277 shutdown_connections 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 -- 172.21.15.174:0/935948277 wait complete. 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 Processor -- start 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.720+0000 7f18f2f80700 1 -- start start 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f2f80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:29.467 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f2f80700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec19a080 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f0d1c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f0d1c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56520/0 (socket says 172.21.15.174:56520) 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f0d1c700 1 -- 172.21.15.174:0/3912708113 learned_addr learned my addr 172.21.15.174:0/3912708113 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f0d1c700 1 -- 172.21.15.174:0/3912708113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0009550 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f0d1c700 1 --2- 172.21.15.174:0/3912708113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec199b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006af0 tx=0x7f18e00119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18e00038b0 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f18e0003a10 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec19a280 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18e0019610 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.721+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec19a720 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.722+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 12) v1 ==== 45910+0+0 (secure 0 0 0) 0x7f18e00198b0 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.722+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec063350 con 0x7f18ec1040c0 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.722+0000 7f18e9ffb700 1 --2- 172.21.15.174:0/3912708113 >> v2:172.21.15.174:6800/2183136494 conn(0x7f18dc038ea0 0x7f18dc03b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:29.468 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.722+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f18e004d2a0 con 0x7f18ec1040c0 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.722+0000 7f18ebfff700 1 --2- 172.21.15.174:0/3912708113 >> v2:172.21.15.174:6800/2183136494 conn(0x7f18dc038ea0 0x7f18dc03b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.723+0000 7f18ebfff700 1 --2- 172.21.15.174:0/3912708113 >> v2:172.21.15.174:6800/2183136494 conn(0x7f18dc038ea0 0x7f18dc03b360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18d8006fd0 tx=0x7f18d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.727+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18e0018dc0 con 0x7f18ec1040c0 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.920+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f18ec19d000 con 0x7f18ec1040c0 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.920+0000 7f18e9ffb700 1 -- 172.21.15.174:0/3912708113 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18e0023390 con 0x7f18ec1040c0 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.922+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 >> v2:172.21.15.174:6800/2183136494 conn(0x7f18dc038ea0 msgr2=0x7f18dc03b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.922+0000 7f18f2f80700 1 --2- 172.21.15.174:0/3912708113 >> v2:172.21.15.174:6800/2183136494 conn(0x7f18dc038ea0 0x7f18dc03b360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18d8006fd0 tx=0x7f18d8006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.922+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 msgr2=0x7f18ec199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.922+0000 7f18f2f80700 1 --2- 172.21.15.174:0/3912708113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec199b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006af0 tx=0x7f18e00119c0 comp rx=0 tx=0).stop 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.922+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 shutdown_connections 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.922+0000 7f18f2f80700 1 --2- 172.21.15.174:0/3912708113 >> v2:172.21.15.174:6800/2183136494 conn(0x7f18dc038ea0 0x7f18dc03b360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.923+0000 7f18f2f80700 1 --2- 172.21.15.174:0/3912708113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18ec1040c0 0x7f18ec199b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.923+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 >> 172.21.15.174:0/3912708113 conn(0x7f18ec0ffa40 msgr2=0x7f18ec100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:29.469 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.923+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 shutdown_connections 2024-02-10T06:39:29.470 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:28.923+0000 7f18f2f80700 1 -- 172.21.15.174:0/3912708113 wait complete. 2024-02-10T06:39:29.474 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T06:39:29.475 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout disabled 2024-02-10T06:39:29.482 INFO:teuthology.orchestra.run.smithi174.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T06:39:29.482 INFO:teuthology.orchestra.run.smithi174.stdout:systemctl: stdout inactive 2024-02-10T06:39:29.482 INFO:teuthology.orchestra.run.smithi174.stdout:firewalld.service is not enabled 2024-02-10T06:39:29.482 INFO:teuthology.orchestra.run.smithi174.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-10T06:39:29.484 INFO:teuthology.orchestra.run.smithi174.stdout:Ceph Dashboard is now available at: 2024-02-10T06:39:29.484 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:29.484 INFO:teuthology.orchestra.run.smithi174.stdout: URL: https://smithi174.front.sepia.ceph.com:8443/ 2024-02-10T06:39:29.484 INFO:teuthology.orchestra.run.smithi174.stdout: User: admin 2024-02-10T06:39:29.484 INFO:teuthology.orchestra.run.smithi174.stdout: Password: 700zom3i30 2024-02-10T06:39:29.484 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:29.485 INFO:teuthology.orchestra.run.smithi174.stdout:Saving cluster configuration to /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config directory 2024-02-10T06:39:29.485 INFO:teuthology.orchestra.run.smithi174.stdout:Enabling autotune for osd_memory_target 2024-02-10T06:39:30.164 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:29 smithi174 ceph-mon[29878]: mgrmap e13: y(active, since 6s) 2024-02-10T06:39:30.764 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c2b8e700 1 Processor -- start 2024-02-10T06:39:30.764 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c2b8e700 1 -- start start 2024-02-10T06:39:30.764 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c2b8e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc1051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:30.764 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c2b8e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1057a0 con 0x7f27bc104db0 2024-02-10T06:39:30.764 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c092a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc1051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:30.764 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c092a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc1051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56524/0 (socket says 172.21.15.174:56524) 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c092a700 1 -- 172.21.15.174:0/3120238528 learned_addr learned my addr 172.21.15.174:0/3120238528 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c092a700 1 -- 172.21.15.174:0/3120238528 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc105fb0 con 0x7f27bc104db0 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.949+0000 7f27c092a700 1 --2- 172.21.15.174:0/3120238528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc1051d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27b00098a0 tx=0x7f27b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d4ca74b51d6e53 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27bb7fe700 1 -- 172.21.15.174:0/3120238528 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b0004160 con 0x7f27bc104db0 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27bb7fe700 1 -- 172.21.15.174:0/3120238528 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f27b00042c0 con 0x7f27bc104db0 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27bb7fe700 1 -- 172.21.15.174:0/3120238528 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b0011960 con 0x7f27bc104db0 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 -- 172.21.15.174:0/3120238528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 msgr2=0x7f27bc1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 --2- 172.21.15.174:0/3120238528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc1051d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27b00098a0 tx=0x7f27b0009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 -- 172.21.15.174:0/3120238528 shutdown_connections 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 --2- 172.21.15.174:0/3120238528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc1051d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 -- 172.21.15.174:0/3120238528 >> 172.21.15.174:0/3120238528 conn(0x7f27bc1003b0 msgr2=0x7f27bc1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 -- 172.21.15.174:0/3120238528 shutdown_connections 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.950+0000 7f27c2b8e700 1 -- 172.21.15.174:0/3120238528 wait complete. 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c2b8e700 1 Processor -- start 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c2b8e700 1 -- start start 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c2b8e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:30.765 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c2b8e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc19a130 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c092a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c092a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56536/0 (socket says 172.21.15.174:56536) 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c092a700 1 -- 172.21.15.174:0/2545006616 learned_addr learned my addr 172.21.15.174:0/2545006616 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c092a700 1 -- 172.21.15.174:0/2545006616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009550 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c092a700 1 --2- 172.21.15.174:0/2545006616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc199bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27b0006af0 tx=0x7f27b0011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b00036a0 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f27b001a430 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.951+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27bc19a330 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.952+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b00193c0 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.952+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc19a7d0 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.952+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f27b0019640 con 0x7f27bc104db0 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.953+0000 7f27b9ffb700 1 --2- 172.21.15.174:0/2545006616 >> v2:172.21.15.174:6800/2183136494 conn(0x7f27ac038f90 0x7f27ac03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:30.766 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.953+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f27b004d120 con 0x7f27bc104db0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.953+0000 7f27bbfff700 1 --2- 172.21.15.174:0/2545006616 >> v2:172.21.15.174:6800/2183136494 conn(0x7f27ac038f90 0x7f27ac03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.953+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27bc193d30 con 0x7f27bc104db0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.953+0000 7f27bbfff700 1 --2- 172.21.15.174:0/2545006616 >> v2:172.21.15.174:6800/2183136494 conn(0x7f27ac038f90 0x7f27ac03b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27a8006fd0 tx=0x7f27a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:29.958+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27b002c3a0 con 0x7f27bc104db0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.149+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f27bc1091e0 con 0x7f27bc104db0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.150+0000 7f27b9ffb700 1 -- 172.21.15.174:0/2545006616 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f27b0010380 con 0x7f27bc104db0 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 >> v2:172.21.15.174:6800/2183136494 conn(0x7f27ac038f90 msgr2=0x7f27ac03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 --2- 172.21.15.174:0/2545006616 >> v2:172.21.15.174:6800/2183136494 conn(0x7f27ac038f90 0x7f27ac03b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27a8006fd0 tx=0x7f27a8006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 msgr2=0x7f27bc199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 --2- 172.21.15.174:0/2545006616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc199bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27b0006af0 tx=0x7f27b0011eb0 comp rx=0 tx=0).stop 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 shutdown_connections 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 --2- 172.21.15.174:0/2545006616 >> v2:172.21.15.174:6800/2183136494 conn(0x7f27ac038f90 0x7f27ac03b450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 --2- 172.21.15.174:0/2545006616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27bc104db0 0x7f27bc199bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:30.767 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 >> 172.21.15.174:0/2545006616 conn(0x7f27bc1003b0 msgr2=0x7f27bc1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:30.768 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 shutdown_connections 2024-02-10T06:39:30.768 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:30.152+0000 7f27c2b8e700 1 -- 172.21.15.174:0/2545006616 wait complete. 2024-02-10T06:39:32.129 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.260+0000 7f9e1d1aa700 1 Processor -- start 2024-02-10T06:39:32.129 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.260+0000 7f9e1d1aa700 1 -- start start 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.260+0000 7f9e1d1aa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18108ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.260+0000 7f9e1d1aa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18078e70 con 0x7f9e181068b0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.260+0000 7f9e16d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18108ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e16d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18108ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56538/0 (socket says 172.21.15.174:56538) 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e16d9d700 1 -- 172.21.15.174:0/2086178157 learned_addr learned my addr 172.21.15.174:0/2086178157 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e16d9d700 1 -- 172.21.15.174:0/2086178157 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e181091e0 con 0x7f9e181068b0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e16d9d700 1 --2- 172.21.15.174:0/2086178157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18108ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e080098a0 tx=0x7f9e08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1a29bf4cef7c8c server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e1659c700 1 -- 172.21.15.174:0/2086178157 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e08004160 con 0x7f9e181068b0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e1659c700 1 -- 172.21.15.174:0/2086178157 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9e080042c0 con 0x7f9e181068b0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.261+0000 7f9e1659c700 1 -- 172.21.15.174:0/2086178157 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e08011960 con 0x7f9e181068b0 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2086178157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 msgr2=0x7f9e18108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 --2- 172.21.15.174:0/2086178157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18108ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e080098a0 tx=0x7f9e08009bb0 comp rx=0 tx=0).stop 2024-02-10T06:39:32.130 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2086178157 shutdown_connections 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 --2- 172.21.15.174:0/2086178157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18108ca0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2086178157 >> 172.21.15.174:0/2086178157 conn(0x7f9e181002c0 msgr2=0x7f9e181026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2086178157 shutdown_connections 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2086178157 wait complete. 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.262+0000 7f9e1d1aa700 1 Processor -- start 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e1d1aa700 1 -- start start 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e1d1aa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e1d1aa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1819a010 con 0x7f9e181068b0 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e16d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e16d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56544/0 (socket says 172.21.15.174:56544) 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e16d9d700 1 -- 172.21.15.174:0/2072353870 learned_addr learned my addr 172.21.15.174:0/2072353870 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e16d9d700 1 -- 172.21.15.174:0/2072353870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08009550 con 0x7f9e181068b0 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e16d9d700 1 --2- 172.21.15.174:0/2072353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18199ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006af0 tx=0x7f9e080119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e080038b0 con 0x7f9e181068b0 2024-02-10T06:39:32.131 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9e08003a10 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1819a210 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e08019610 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.263+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1819a6b0 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.264+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f9e080198b0 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.264+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e18109fc0 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.265+0000 7f9e14d99700 1 --2- 172.21.15.174:0/2072353870 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9e04038f40 0x7f9e0403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.265+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f9e0804d380 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.265+0000 7f9e0e59c700 1 --2- 172.21.15.174:0/2072353870 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9e04038f40 0x7f9e0403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.265+0000 7f9e0e59c700 1 --2- 172.21.15.174:0/2072353870 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9e04038f40 0x7f9e0403b400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e00006fd0 tx=0x7f9e00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.269+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e08018dc0 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.568+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f9e181047a0 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.571+0000 7f9e14d99700 1 -- 172.21.15.174:0/2072353870 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f9e08023390 con 0x7f9e181068b0 2024-02-10T06:39:32.132 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9e04038f40 msgr2=0x7f9e0403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 --2- 172.21.15.174:0/2072353870 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9e04038f40 0x7f9e0403b400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e00006fd0 tx=0x7f9e00006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 msgr2=0x7f9e18199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 --2- 172.21.15.174:0/2072353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18199ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006af0 tx=0x7f9e080119c0 comp rx=0 tx=0).stop 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 shutdown_connections 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 --2- 172.21.15.174:0/2072353870 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9e04038f40 0x7f9e0403b400 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 --2- 172.21.15.174:0/2072353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e181068b0 0x7f9e18199ad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 >> 172.21.15.174:0/2072353870 conn(0x7f9e181002c0 msgr2=0x7f9e18100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 shutdown_connections 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr 2024-02-10T06:39:31.573+0000 7f9e1d1aa700 1 -- 172.21.15.174:0/2072353870 wait complete. 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout: sudo /sbin/cephadm shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-10T06:39:32.133 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout:Or, if you are only running a single cluster on this host: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: sudo /sbin/cephadm shell 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: ceph telemetry on 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout:For more information see: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:32.134 INFO:teuthology.orchestra.run.smithi174.stdout:Bootstrap complete. 2024-02-10T06:39:32.171 INFO:tasks.cephadm:Fetching config... 2024-02-10T06:39:32.172 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:39:32.172 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-10T06:39:32.187 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-10T06:39:32.188 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:39:32.188 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-10T06:39:32.242 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-10T06:39:32.243 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:39:32.243 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-10T06:39:32.309 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-10T06:39:32.310 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:39:32.310 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-10T06:39:32.365 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-10T06:39:32.365 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKXkgq4Zsitu3iChjhA6edpwDy5zmESu4bPYzqyR1WczQ8o2Y3PFgDMPCTmELJQH7A7mO3sUAYjBm830FGEzHkLROMjcvScUQ4wlbsvRDdAsTBiqy1oJIHjtrvBF76sFNfJGj6g8yAL+Msg+QJvZAcXyeye+OFuwSIE4VgvK3fdXeDE8lRNg7dSg6KPTzVYWEXrvWe5Ttg05SAn9gIBbRgx7D8Jm3Alh8vthW3ib9Exx/6mLE2YhvGi3T49Vtblp/5tL93a1rjpfABalMB/+vG24i+ZGQG/JnqBpi8ZS93OJCHN2v3v6xh1UG82QeS3aTuofPYTLsMDeA+gM7NTvb39WCeyJFGimPllri8KFGtLXXKnOCt+kGM22+dqHoCGcI5EaX93BWt8uwNEtAdaHkH0/yNOBveBzjg88LYUFXVenMrqkXnWmBLU6n2nwo6k/2Zcr/pKcyWmVgPYYOGNy7vz/yygSHICizhGj7v30KySl+3NiLCEAelyygXKOzbnws= ceph-df845a14-c7de-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T06:39:32.442 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKXkgq4Zsitu3iChjhA6edpwDy5zmESu4bPYzqyR1WczQ8o2Y3PFgDMPCTmELJQH7A7mO3sUAYjBm830FGEzHkLROMjcvScUQ4wlbsvRDdAsTBiqy1oJIHjtrvBF76sFNfJGj6g8yAL+Msg+QJvZAcXyeye+OFuwSIE4VgvK3fdXeDE8lRNg7dSg6KPTzVYWEXrvWe5Ttg05SAn9gIBbRgx7D8Jm3Alh8vthW3ib9Exx/6mLE2YhvGi3T49Vtblp/5tL93a1rjpfABalMB/+vG24i+ZGQG/JnqBpi8ZS93OJCHN2v3v6xh1UG82QeS3aTuofPYTLsMDeA+gM7NTvb39WCeyJFGimPllri8KFGtLXXKnOCt+kGM22+dqHoCGcI5EaX93BWt8uwNEtAdaHkH0/yNOBveBzjg88LYUFXVenMrqkXnWmBLU6n2nwo6k/2Zcr/pKcyWmVgPYYOGNy7vz/yygSHICizhGj7v30KySl+3NiLCEAelyygXKOzbnws= ceph-df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:39:32.456 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKXkgq4Zsitu3iChjhA6edpwDy5zmESu4bPYzqyR1WczQ8o2Y3PFgDMPCTmELJQH7A7mO3sUAYjBm830FGEzHkLROMjcvScUQ4wlbsvRDdAsTBiqy1oJIHjtrvBF76sFNfJGj6g8yAL+Msg+QJvZAcXyeye+OFuwSIE4VgvK3fdXeDE8lRNg7dSg6KPTzVYWEXrvWe5Ttg05SAn9gIBbRgx7D8Jm3Alh8vthW3ib9Exx/6mLE2YhvGi3T49Vtblp/5tL93a1rjpfABalMB/+vG24i+ZGQG/JnqBpi8ZS93OJCHN2v3v6xh1UG82QeS3aTuofPYTLsMDeA+gM7NTvb39WCeyJFGimPllri8KFGtLXXKnOCt+kGM22+dqHoCGcI5EaX93BWt8uwNEtAdaHkH0/yNOBveBzjg88LYUFXVenMrqkXnWmBLU6n2nwo6k/2Zcr/pKcyWmVgPYYOGNy7vz/yygSHICizhGj7v30KySl+3NiLCEAelyygXKOzbnws= ceph-df845a14-c7de-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T06:39:32.502 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKXkgq4Zsitu3iChjhA6edpwDy5zmESu4bPYzqyR1WczQ8o2Y3PFgDMPCTmELJQH7A7mO3sUAYjBm830FGEzHkLROMjcvScUQ4wlbsvRDdAsTBiqy1oJIHjtrvBF76sFNfJGj6g8yAL+Msg+QJvZAcXyeye+OFuwSIE4VgvK3fdXeDE8lRNg7dSg6KPTzVYWEXrvWe5Ttg05SAn9gIBbRgx7D8Jm3Alh8vthW3ib9Exx/6mLE2YhvGi3T49Vtblp/5tL93a1rjpfABalMB/+vG24i+ZGQG/JnqBpi8ZS93OJCHN2v3v6xh1UG82QeS3aTuofPYTLsMDeA+gM7NTvb39WCeyJFGimPllri8KFGtLXXKnOCt+kGM22+dqHoCGcI5EaX93BWt8uwNEtAdaHkH0/yNOBveBzjg88LYUFXVenMrqkXnWmBLU6n2nwo6k/2Zcr/pKcyWmVgPYYOGNy7vz/yygSHICizhGj7v30KySl+3NiLCEAelyygXKOzbnws= ceph-df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:39:32.517 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-10T06:39:32.692 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2072353870' entity='client.admin' 2024-02-10T06:39:32.757 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:39:35.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 -- 172.21.15.174:0/2187430973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 msgr2=0x7fbc8c100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:35.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 --2- 172.21.15.174:0/2187430973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c100ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009910 tx=0x7fbc7c009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:35.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 -- 172.21.15.174:0/2187430973 shutdown_connections 2024-02-10T06:39:35.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 --2- 172.21.15.174:0/2187430973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c100ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:35.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 -- 172.21.15.174:0/2187430973 >> 172.21.15.174:0/2187430973 conn(0x7fbc8c0fbc80 msgr2=0x7fbc8c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:35.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 -- 172.21.15.174:0/2187430973 shutdown_connections 2024-02-10T06:39:35.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 -- 172.21.15.174:0/2187430973 wait complete. 2024-02-10T06:39:35.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.118+0000 7fbc9135d700 1 Processor -- start 2024-02-10T06:39:35.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc9135d700 1 -- start start 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc9135d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc9135d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c196910 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56560/0 (socket says 172.21.15.174:56560) 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc8affd700 1 -- 172.21.15.174:0/1239112683 learned_addr learned my addr 172.21.15.174:0/1239112683 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc8affd700 1 -- 172.21.15.174:0/1239112683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c0095f0 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc8affd700 1 --2- 172.21.15.174:0/1239112683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c1963d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0057a0 tx=0x7fbc7c004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c01d070 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fbc7c022470 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c196b10 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c00f5f0 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.119+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c196fb0 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.120+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fbc7c00f830 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.120+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c1901c0 con 0x7fbc8c1006c0 2024-02-10T06:39:35.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.120+0000 7fbc88ff9700 1 --2- 172.21.15.174:0/1239112683 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbc78038f40 0x7fbc7803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:35.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.121+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fbc7c04d210 con 0x7fbc8c1006c0 2024-02-10T06:39:35.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.121+0000 7fbc827fc700 1 --2- 172.21.15.174:0/1239112683 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbc78038f40 0x7fbc7803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:35.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.121+0000 7fbc827fc700 1 --2- 172.21.15.174:0/1239112683 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbc78038f40 0x7fbc7803b400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006fd0 tx=0x7fbc74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:35.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.126+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc7c02a770 con 0x7fbc8c1006c0 2024-02-10T06:39:35.317 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.316+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fbc8c105dc0 con 0x7fbc8c1006c0 2024-02-10T06:39:35.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.321+0000 7fbc88ff9700 1 -- 172.21.15.174:0/1239112683 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbc7c027070 con 0x7fbc8c1006c0 2024-02-10T06:39:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbc78038f40 msgr2=0x7fbc7803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 --2- 172.21.15.174:0/1239112683 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbc78038f40 0x7fbc7803b400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006fd0 tx=0x7fbc74006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 msgr2=0x7fbc8c1963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 --2- 172.21.15.174:0/1239112683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c1963d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0057a0 tx=0x7fbc7c004f20 comp rx=0 tx=0).stop 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 shutdown_connections 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 --2- 172.21.15.174:0/1239112683 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbc78038f40 0x7fbc7803b400 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 --2- 172.21.15.174:0/1239112683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc8c1006c0 0x7fbc8c1963d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 >> 172.21.15.174:0/1239112683 conn(0x7fbc8c0fbc80 msgr2=0x7fbc8c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 shutdown_connections 2024-02-10T06:39:35.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:35.324+0000 7fbc9135d700 1 -- 172.21.15.174:0/1239112683 wait complete. 2024-02-10T06:39:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:39:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:35.860 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-10T06:39:35.861 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-10T06:39:36.095 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:39:36.468 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1239112683' entity='client.admin' 2024-02-10T06:39:36.468 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:36 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:36.468 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:36 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 -- 172.21.15.174:0/523487277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 msgr2=0x7fd548100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 --2- 172.21.15.174:0/523487277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548100a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd538009910 tx=0x7fd538009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 -- 172.21.15.174:0/523487277 shutdown_connections 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 --2- 172.21.15.174:0/523487277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548100a70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 -- 172.21.15.174:0/523487277 >> 172.21.15.174:0/523487277 conn(0x7fd5480fbc10 msgr2=0x7fd5480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 -- 172.21.15.174:0/523487277 shutdown_connections 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.540+0000 7fd54dc84700 1 -- 172.21.15.174:0/523487277 wait complete. 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd54dc84700 1 Processor -- start 2024-02-10T06:39:37.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd54dc84700 1 -- start start 2024-02-10T06:39:37.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd54dc84700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:37.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd54dc84700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548196950 con 0x7fd548100650 2024-02-10T06:39:37.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd5477fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:37.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd5477fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:44094/0 (socket says 172.21.15.174:44094) 2024-02-10T06:39:37.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.541+0000 7fd5477fe700 1 -- 172.21.15.174:0/410396087 learned_addr learned my addr 172.21.15.174:0/410396087 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd5477fe700 1 -- 172.21.15.174:0/410396087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380095f0 con 0x7fd548100650 2024-02-10T06:39:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd5477fe700 1 --2- 172.21.15.174:0/410396087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548196410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd538005160 tx=0x7fd538004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53801d070 con 0x7fd548100650 2024-02-10T06:39:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd53800bc00 con 0x7fd548100650 2024-02-10T06:39:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd548196b50 con 0x7fd548100650 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5380038c0 con 0x7fd548100650 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.542+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd548196ff0 con 0x7fd548100650 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.543+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5481901c0 con 0x7fd548100650 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.543+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fd538003b30 con 0x7fd548100650 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.543+0000 7fd544ff9700 1 --2- 172.21.15.174:0/410396087 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd530038f90 0x7fd53003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.543+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd53804cea0 con 0x7fd548100650 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.543+0000 7fd546ffd700 1 --2- 172.21.15.174:0/410396087 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd530038f90 0x7fd53003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.543+0000 7fd546ffd700 1 --2- 172.21.15.174:0/410396087 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd530038f90 0x7fd53003b450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd53c006fd0 tx=0x7fd53c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:37.548 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.548+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd538004390 con 0x7fd548100650 2024-02-10T06:39:37.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.748+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd548105d40 con 0x7fd530038f90 2024-02-10T06:39:37.753 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.753+0000 7fd544ff9700 1 -- 172.21.15.174:0/410396087 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd548105d40 con 0x7fd530038f90 2024-02-10T06:39:37.755 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd530038f90 msgr2=0x7fd53003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 --2- 172.21.15.174:0/410396087 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd530038f90 0x7fd53003b450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd53c006fd0 tx=0x7fd53c006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 msgr2=0x7fd548196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 --2- 172.21.15.174:0/410396087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548196410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd538005160 tx=0x7fd538004f00 comp rx=0 tx=0).stop 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 shutdown_connections 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 --2- 172.21.15.174:0/410396087 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd530038f90 0x7fd53003b450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 --2- 172.21.15.174:0/410396087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd548100650 0x7fd548196410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 >> 172.21.15.174:0/410396087 conn(0x7fd5480fbc10 msgr2=0x7fd548104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 shutdown_connections 2024-02-10T06:39:37.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:37.755+0000 7fd54dc84700 1 -- 172.21.15.174:0/410396087 wait complete. 2024-02-10T06:39:38.519 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-02-10T06:39:38.519 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:39:38.520 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-02-10T06:39:38.537 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:39:38.537 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:39:38.595 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-02-10T06:39:38.595 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch host add smithi184 2024-02-10T06:39:38.830 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:39:39.049 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:39:39.050 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:39:39.050 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:39.050 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:39.050 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:40.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 -- 172.21.15.174:0/3276571151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 msgr2=0x7efd9c106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:40.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 --2- 172.21.15.174:0/3276571151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c106a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd90009910 tx=0x7efd90009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:40.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 -- 172.21.15.174:0/3276571151 shutdown_connections 2024-02-10T06:39:40.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 --2- 172.21.15.174:0/3276571151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c106a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:40.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 -- 172.21.15.174:0/3276571151 >> 172.21.15.174:0/3276571151 conn(0x7efd9c0fc130 msgr2=0x7efd9c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:40.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 -- 172.21.15.174:0/3276571151 shutdown_connections 2024-02-10T06:39:40.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.355+0000 7efda0bb4700 1 -- 172.21.15.174:0/3276571151 wait complete. 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efda0bb4700 1 Processor -- start 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efda0bb4700 1 -- start start 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efda0bb4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efda0bb4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c06a930 con 0x7efd9c106690 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efd9a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efd9a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:44116/0 (socket says 172.21.15.174:44116) 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.356+0000 7efd9a59c700 1 -- 172.21.15.174:0/1578604976 learned_addr learned my addr 172.21.15.174:0/1578604976 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:40.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efd9a59c700 1 -- 172.21.15.174:0/1578604976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900095f0 con 0x7efd9c106690 2024-02-10T06:39:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efd9a59c700 1 --2- 172.21.15.174:0/1578604976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c195360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd90000c00 tx=0x7efd9000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9001d070 con 0x7efd9c106690 2024-02-10T06:39:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7efd90004500 con 0x7efd9c106690 2024-02-10T06:39:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd90003ae0 con 0x7efd9c106690 2024-02-10T06:39:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c1958a0 con 0x7efd9c106690 2024-02-10T06:39:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.357+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c195da0 con 0x7efd9c106690 2024-02-10T06:39:40.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.358+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7efd9002b440 con 0x7efd9c106690 2024-02-10T06:39:40.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.358+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c105980 con 0x7efd9c106690 2024-02-10T06:39:40.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.358+0000 7efd8b7fe700 1 --2- 172.21.15.174:0/1578604976 >> v2:172.21.15.174:6800/2183136494 conn(0x7efd84038f40 0x7efd8403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:40.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.358+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7efd9004cf60 con 0x7efd9c106690 2024-02-10T06:39:40.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.358+0000 7efd99d9b700 1 --2- 172.21.15.174:0/1578604976 >> v2:172.21.15.174:6800/2183136494 conn(0x7efd84038f40 0x7efd8403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:40.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.358+0000 7efd99d9b700 1 --2- 172.21.15.174:0/1578604976 >> v2:172.21.15.174:6800/2183136494 conn(0x7efd84038f40 0x7efd8403b400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd8c006fd0 tx=0x7efd8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:40.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.363+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd90015740 con 0x7efd9c106690 2024-02-10T06:39:40.561 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:40.560+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}) v1 -- 0x7efd9c06a120 con 0x7efd84038f40 2024-02-10T06:39:41.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:41 smithi174 ceph-mon[29878]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:41.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:41 smithi174 ceph-mon[29878]: Deploying cephadm binary to smithi184 2024-02-10T06:39:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:43 smithi174 ceph-mon[29878]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:44.967 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.966+0000 7efd8b7fe700 1 -- 172.21.15.174:0/1578604976 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efd9c06a120 con 0x7efd84038f40 2024-02-10T06:39:44.967 INFO:teuthology.orchestra.run.smithi174.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 >> v2:172.21.15.174:6800/2183136494 conn(0x7efd84038f40 msgr2=0x7efd8403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 --2- 172.21.15.174:0/1578604976 >> v2:172.21.15.174:6800/2183136494 conn(0x7efd84038f40 0x7efd8403b400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd8c006fd0 tx=0x7efd8c006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 msgr2=0x7efd9c195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 --2- 172.21.15.174:0/1578604976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c195360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd90000c00 tx=0x7efd9000f740 comp rx=0 tx=0).stop 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 shutdown_connections 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 --2- 172.21.15.174:0/1578604976 >> v2:172.21.15.174:6800/2183136494 conn(0x7efd84038f40 0x7efd8403b400 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 --2- 172.21.15.174:0/1578604976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd9c106690 0x7efd9c195360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 >> 172.21.15.174:0/1578604976 conn(0x7efd9c0fc130 msgr2=0x7efd9c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:44.970 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 shutdown_connections 2024-02-10T06:39:44.970 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:44.968+0000 7efda0bb4700 1 -- 172.21.15.174:0/1578604976 wait complete. 2024-02-10T06:39:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:45 smithi174 ceph-mon[29878]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:45.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T06:39:45.777 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:39:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:46 smithi174 ceph-mon[29878]: Added host smithi184 2024-02-10T06:39:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.346+0000 7fe20b704700 1 -- 172.21.15.174:0/604817056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 msgr2=0x7fe204102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.346+0000 7fe20b704700 1 --2- 172.21.15.174:0/604817056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe204102c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe200009910 tx=0x7fe200009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.346+0000 7fe20b704700 1 -- 172.21.15.174:0/604817056 shutdown_connections 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.346+0000 7fe20b704700 1 --2- 172.21.15.174:0/604817056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe204102c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.346+0000 7fe20b704700 1 -- 172.21.15.174:0/604817056 >> 172.21.15.174:0/604817056 conn(0x7fe2040fc0d0 msgr2=0x7fe2040fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.346+0000 7fe20b704700 1 -- 172.21.15.174:0/604817056 shutdown_connections 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe20b704700 1 -- 172.21.15.174:0/604817056 wait complete. 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe20b704700 1 Processor -- start 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe20b704700 1 -- start start 2024-02-10T06:39:47.349 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe20b704700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe20419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe20b704700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20419c980 con 0x7fe2041028a0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe2094a0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe20419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe2094a0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe20419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53168/0 (socket says 172.21.15.174:53168) 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.347+0000 7fe2094a0700 1 -- 172.21.15.174:0/4057577903 learned_addr learned my addr 172.21.15.174:0/4057577903 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe2094a0700 1 -- 172.21.15.174:0/4057577903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000095f0 con 0x7fe2041028a0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe2094a0700 1 --2- 172.21.15.174:0/4057577903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe20419c440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe20000f690 tx=0x7fe20000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe20001d070 con 0x7fe2041028a0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe200004500 con 0x7fe2041028a0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20419cb80 con 0x7fe2041028a0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe200003c10 con 0x7fe2041028a0 2024-02-10T06:39:47.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.348+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20419cfa0 con 0x7fe2041028a0 2024-02-10T06:39:47.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.349+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fe200004050 con 0x7fe2041028a0 2024-02-10T06:39:47.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.349+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe204063350 con 0x7fe2041028a0 2024-02-10T06:39:47.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.349+0000 7fe1fa7fc700 1 --2- 172.21.15.174:0/4057577903 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1f0038f90 0x7fe1f003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:47.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.349+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe20004cca0 con 0x7fe2041028a0 2024-02-10T06:39:47.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.349+0000 7fe208c9f700 1 --2- 172.21.15.174:0/4057577903 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1f0038f90 0x7fe1f003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:47.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.349+0000 7fe208c9f700 1 --2- 172.21.15.174:0/4057577903 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1f0038f90 0x7fe1f003b450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006fd0 tx=0x7fe1f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:47.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.354+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe200015530 con 0x7fe2041028a0 2024-02-10T06:39:47.555 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.554+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe20410b2c0 con 0x7fe1f0038f90 2024-02-10T06:39:47.556 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.555+0000 7fe1fa7fc700 1 -- 172.21.15.174:0/4057577903 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fe20410b2c0 con 0x7fe1f0038f90 2024-02-10T06:39:47.556 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:39:47.556 INFO:teuthology.orchestra.run.smithi174.stdout:[{"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1f0038f90 msgr2=0x7fe1f003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 --2- 172.21.15.174:0/4057577903 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1f0038f90 0x7fe1f003b450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006fd0 tx=0x7fe1f4006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 msgr2=0x7fe20419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 --2- 172.21.15.174:0/4057577903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe20419c440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe20000f690 tx=0x7fe20000f770 comp rx=0 tx=0).stop 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 shutdown_connections 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 --2- 172.21.15.174:0/4057577903 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1f0038f90 0x7fe1f003b450 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 --2- 172.21.15.174:0/4057577903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2041028a0 0x7fe20419c440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 >> 172.21.15.174:0/4057577903 conn(0x7fe2040fc0d0 msgr2=0x7fe2040fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:47.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 shutdown_connections 2024-02-10T06:39:47.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:47.558+0000 7fe20b704700 1 -- 172.21.15.174:0/4057577903 wait complete. 2024-02-10T06:39:47.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:47 smithi174 ceph-mon[29878]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:48.141 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-10T06:39:48.142 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd crush tunables default 2024-02-10T06:39:48.375 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:39:48.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:48.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:48 smithi174 ceph-mon[29878]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:39:48.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:48 smithi174 ceph-mon[29878]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 -- 172.21.15.174:0/4055150638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 msgr2=0x7fb6b8107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 --2- 172.21.15.174:0/4055150638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b8107ac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009910 tx=0x7fb6a8009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 -- 172.21.15.174:0/4055150638 shutdown_connections 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 --2- 172.21.15.174:0/4055150638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b8107ac0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 -- 172.21.15.174:0/4055150638 >> 172.21.15.174:0/4055150638 conn(0x7fb6b80fd1f0 msgr2=0x7fb6b80ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 -- 172.21.15.174:0/4055150638 shutdown_connections 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.908+0000 7fb6be4c0700 1 -- 172.21.15.174:0/4055150638 wait complete. 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6be4c0700 1 Processor -- start 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6be4c0700 1 -- start start 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6be4c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b81966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:49.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6be4c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8196c00 con 0x7fb6b81076e0 2024-02-10T06:39:49.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b81966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:49.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b81966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53184/0 (socket says 172.21.15.174:53184) 2024-02-10T06:39:49.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6b7fff700 1 -- 172.21.15.174:0/661684146 learned_addr learned my addr 172.21.15.174:0/661684146 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:39:49.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6b7fff700 1 -- 172.21.15.174:0/661684146 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80095f0 con 0x7fb6b81076e0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.909+0000 7fb6b7fff700 1 --2- 172.21.15.174:0/661684146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b81966c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800f690 tx=0x7fb6a800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.910+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6a801d070 con 0x7fb6b81076e0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.910+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb6a8004500 con 0x7fb6b81076e0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.910+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b8196e60 con 0x7fb6b81076e0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.910+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8003b30 con 0x7fb6b81076e0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.910+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b81972e0 con 0x7fb6b81076e0 2024-02-10T06:39:49.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.911+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fb6a8004050 con 0x7fb6b81076e0 2024-02-10T06:39:49.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.911+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b8063350 con 0x7fb6b81076e0 2024-02-10T06:39:49.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.911+0000 7fb6b57fa700 1 --2- 172.21.15.174:0/661684146 >> v2:172.21.15.174:6800/2183136494 conn(0x7fb6a0038f40 0x7fb6a003b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:49.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.911+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fb6a804cc20 con 0x7fb6b81076e0 2024-02-10T06:39:49.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.911+0000 7fb6b77fe700 1 --2- 172.21.15.174:0/661684146 >> v2:172.21.15.174:6800/2183136494 conn(0x7fb6a0038f40 0x7fb6a003b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:49.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.911+0000 7fb6b77fe700 1 --2- 172.21.15.174:0/661684146 >> v2:172.21.15.174:6800/2183136494 conn(0x7fb6a0038f40 0x7fb6a003b400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac006fd0 tx=0x7fb6ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:49.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:49.916+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6a802b440 con 0x7fb6b81076e0 2024-02-10T06:39:50.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.116+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb6b8197a20 con 0x7fb6b81076e0 2024-02-10T06:39:50.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.167+0000 7fb6b57fa700 1 -- 172.21.15.174:0/661684146 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb6a8026090 con 0x7fb6b81076e0 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.169+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 >> v2:172.21.15.174:6800/2183136494 conn(0x7fb6a0038f40 msgr2=0x7fb6a003b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.169+0000 7fb6be4c0700 1 --2- 172.21.15.174:0/661684146 >> v2:172.21.15.174:6800/2183136494 conn(0x7fb6a0038f40 0x7fb6a003b400 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac006fd0 tx=0x7fb6ac006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.169+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 msgr2=0x7fb6b81966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.169+0000 7fb6be4c0700 1 --2- 172.21.15.174:0/661684146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b81966c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800f690 tx=0x7fb6a800f770 comp rx=0 tx=0).stop 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.170+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 shutdown_connections 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.170+0000 7fb6be4c0700 1 --2- 172.21.15.174:0/661684146 >> v2:172.21.15.174:6800/2183136494 conn(0x7fb6a0038f40 0x7fb6a003b400 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.170+0000 7fb6be4c0700 1 --2- 172.21.15.174:0/661684146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b81076e0 0x7fb6b81966c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.170+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 >> 172.21.15.174:0/661684146 conn(0x7fb6b80fd1f0 msgr2=0x7fb6b8109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.170+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 shutdown_connections 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:39:50.170+0000 7fb6be4c0700 1 -- 172.21.15.174:0/661684146 wait complete. 2024-02-10T06:39:50.171 INFO:teuthology.orchestra.run.smithi174.stderr:adjusted tunables profile to default 2024-02-10T06:39:50.458 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/661684146' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T06:39:50.731 INFO:tasks.cephadm:Adding mon.a on smithi174 2024-02-10T06:39:50.731 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-02-10T06:39:50.731 INFO:tasks.cephadm:Adding mon.b on smithi184 2024-02-10T06:39:50.731 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply mon '3;smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b' 2024-02-10T06:39:50.967 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-10T06:39:51.013 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-10T06:39:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/661684146' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T06:39:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:39:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:51.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:51.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:51.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:39:51.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:51.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:39:52.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 -- 172.21.15.184:0/3695635177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 msgr2=0x7f21b8107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:52.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 --2- 172.21.15.184:0/3695635177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b8107ac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009910 tx=0x7f21b4009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:52.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 -- 172.21.15.184:0/3695635177 shutdown_connections 2024-02-10T06:39:52.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 --2- 172.21.15.184:0/3695635177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b8107ac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:52.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 -- 172.21.15.184:0/3695635177 >> 172.21.15.184:0/3695635177 conn(0x7f21b80fd1f0 msgr2=0x7f21b80ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:52.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 -- 172.21.15.184:0/3695635177 shutdown_connections 2024-02-10T06:39:52.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.502+0000 7f21bfa59700 1 -- 172.21.15.184:0/3695635177 wait complete. 2024-02-10T06:39:52.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bfa59700 1 Processor -- start 2024-02-10T06:39:52.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bfa59700 1 -- start start 2024-02-10T06:39:52.506 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bfa59700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:52.506 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bfa59700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b819ca70 con 0x7f21b81076e0 2024-02-10T06:39:52.506 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bd7f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:52.506 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bd7f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.184:52356/0 (socket says 172.21.15.184:52356) 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.503+0000 7f21bd7f5700 1 -- 172.21.15.184:0/3457367844 learned_addr learned my addr 172.21.15.184:0/3457367844 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21bd7f5700 1 -- 172.21.15.184:0/3457367844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b40095f0 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21bd7f5700 1 --2- 172.21.15.184:0/3457367844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b819c530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21b400f690 tx=0x7f21b400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b401d070 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f21b4004500 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b819ccd0 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21b4003b30 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.504+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b81966e0 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.505+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b8102e20 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.505+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f21b4004050 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.505+0000 7f21ae7fc700 1 --2- 172.21.15.184:0/3457367844 >> v2:172.21.15.174:6800/2183136494 conn(0x7f21a4038f40 0x7f21a403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.505+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f21b404cc20 con 0x7f21b81076e0 2024-02-10T06:39:52.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.505+0000 7f21bcff4700 1 --2- 172.21.15.184:0/3457367844 >> v2:172.21.15.174:6800/2183136494 conn(0x7f21a4038f40 0x7f21a403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:52.508 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.506+0000 7f21bcff4700 1 --2- 172.21.15.184:0/3457367844 >> v2:172.21.15.174:6800/2183136494 conn(0x7f21a4038f40 0x7f21a403b400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006fd0 tx=0x7f21a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:52.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.510+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21b4015530 con 0x7f21b81076e0 2024-02-10T06:39:52.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.711+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f21b8197ba0 con 0x7f21a4038f40 2024-02-10T06:39:52.718 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.718+0000 7f21ae7fc700 1 -- 172.21.15.184:0/3457367844 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f21b8197ba0 con 0x7f21a4038f40 2024-02-10T06:39:52.719 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 >> v2:172.21.15.174:6800/2183136494 conn(0x7f21a4038f40 msgr2=0x7f21a403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 --2- 172.21.15.184:0/3457367844 >> v2:172.21.15.174:6800/2183136494 conn(0x7f21a4038f40 0x7f21a403b400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21a8006fd0 tx=0x7f21a8006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 msgr2=0x7f21b819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 --2- 172.21.15.184:0/3457367844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b819c530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21b400f690 tx=0x7f21b400f770 comp rx=0 tx=0).stop 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 shutdown_connections 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 --2- 172.21.15.184:0/3457367844 >> v2:172.21.15.174:6800/2183136494 conn(0x7f21a4038f40 0x7f21a403b400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 --2- 172.21.15.184:0/3457367844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21b81076e0 0x7f21b819c530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:52.721 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 >> 172.21.15.184:0/3457367844 conn(0x7f21b80fd1f0 msgr2=0x7f21b8109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:52.722 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 shutdown_connections 2024-02-10T06:39:52.722 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:52.720+0000 7f21bfa59700 1 -- 172.21.15.184:0/3457367844 wait complete. 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:53.389 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.c.service 2024-02-10T06:39:53.392 DEBUG:teuthology.orchestra.run.smithi184:mon.b> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.b.service 2024-02-10T06:39:53.394 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T06:39:53.394 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:39:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:54.911 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:54 smithi174 ceph-mon[29878]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:54.911 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:54 smithi174 ceph-mon[29878]: Saving service mon spec with placement smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b;count:3 2024-02-10T06:39:54.911 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:54 smithi174 ceph-mon[29878]: Deploying daemon mon.b on smithi184 2024-02-10T06:39:55.465 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:39:55.831 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:55 smithi174 ceph-mon[29878]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:56 smithi184 systemd[1]: Starting Ceph mon.b for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:39:57.190 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 -- 172.21.15.184:0/4007549604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 msgr2=0x7efe28102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:57.190 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 --2- 172.21.15.184:0/4007549604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe28102d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efe18009910 tx=0x7efe18009c20 comp rx=0 tx=0).stop 2024-02-10T06:39:57.190 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 -- 172.21.15.184:0/4007549604 shutdown_connections 2024-02-10T06:39:57.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 --2- 172.21.15.184:0/4007549604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe28102d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:57.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 -- 172.21.15.184:0/4007549604 >> 172.21.15.184:0/4007549604 conn(0x7efe280fc1d0 msgr2=0x7efe280fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:57.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 -- 172.21.15.184:0/4007549604 shutdown_connections 2024-02-10T06:39:57.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.188+0000 7efe2f2b8700 1 -- 172.21.15.184:0/4007549604 wait complete. 2024-02-10T06:39:57.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.189+0000 7efe2f2b8700 1 Processor -- start 2024-02-10T06:39:57.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.189+0000 7efe2f2b8700 1 -- start start 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.190+0000 7efe2f2b8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe2819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.190+0000 7efe2f2b8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819ec70 con 0x7efe281029a0 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.190+0000 7efe2d054700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe2819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.190+0000 7efe2d054700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe2819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.184:52378/0 (socket says 172.21.15.184:52378) 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.190+0000 7efe2d054700 1 -- 172.21.15.184:0/3646760626 learned_addr learned my addr 172.21.15.184:0/3646760626 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.190+0000 7efe2d054700 1 -- 172.21.15.184:0/3646760626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180095f0 con 0x7efe281029a0 2024-02-10T06:39:57.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.191+0000 7efe2d054700 1 --2- 172.21.15.184:0/3646760626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe2819e730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe1800f690 tx=0x7efe1800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.191+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe1801d070 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.191+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2819ee70 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.191+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7efe18004500 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.191+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe18003c10 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.191+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2819f290 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.192+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28063350 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.192+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7efe1802b440 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.192+0000 7efe1e7fc700 1 --2- 172.21.15.184:0/3646760626 >> v2:172.21.15.174:6800/2183136494 conn(0x7efe14038f90 0x7efe1403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.192+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7efe1804ccb0 con 0x7efe281029a0 2024-02-10T06:39:57.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.192+0000 7efe2c853700 1 --2- 172.21.15.184:0/3646760626 >> v2:172.21.15.174:6800/2183136494 conn(0x7efe14038f90 0x7efe1403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:39:57.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.193+0000 7efe2c853700 1 --2- 172.21.15.184:0/3646760626 >> v2:172.21.15.174:6800/2183136494 conn(0x7efe14038f90 0x7efe1403b450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe24006fd0 tx=0x7efe24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:39:57.198 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.197+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe18015760 con 0x7efe281029a0 2024-02-10T06:39:57.481 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.479+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efe2810b3d0 con 0x7efe281029a0 2024-02-10T06:39:57.481 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:39:57.481 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"df845a14-c7de-11ee-95b8-87774f69a715","modified":"2024-02-10T06:38:12.344337Z","created":"2024-02-10T06:38:12.344337Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-10T06:39:57.482 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.480+0000 7efe1e7fc700 1 -- 172.21.15.184:0/3646760626 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7efe18026030 con 0x7efe281029a0 2024-02-10T06:39:57.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 >> v2:172.21.15.174:6800/2183136494 conn(0x7efe14038f90 msgr2=0x7efe1403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:57.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 --2- 172.21.15.184:0/3646760626 >> v2:172.21.15.174:6800/2183136494 conn(0x7efe14038f90 0x7efe1403b450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe24006fd0 tx=0x7efe24006e40 comp rx=0 tx=0).stop 2024-02-10T06:39:57.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 msgr2=0x7efe2819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:39:57.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 --2- 172.21.15.184:0/3646760626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe2819e730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe1800f690 tx=0x7efe1800f770 comp rx=0 tx=0).stop 2024-02-10T06:39:57.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 shutdown_connections 2024-02-10T06:39:57.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 --2- 172.21.15.184:0/3646760626 >> v2:172.21.15.174:6800/2183136494 conn(0x7efe14038f90 0x7efe1403b450 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:57.484 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 --2- 172.21.15.184:0/3646760626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe281029a0 0x7efe2819e730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:39:57.484 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 >> 172.21.15.184:0/3646760626 conn(0x7efe280fc1d0 msgr2=0x7efe280fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:39:57.484 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 shutdown_connections 2024-02-10T06:39:57.484 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:39:57.482+0000 7efe2f2b8700 1 -- 172.21.15.184:0/3646760626 wait complete. 2024-02-10T06:39:57.484 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2024-02-10T06:39:57.572 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 podman[33443]: 2024-02-10 06:39:57.201584204 +0000 UTC m=+0.019774467 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:39:57.572 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 podman[33443]: 2024-02-10 06:39:57.352509956 +0000 UTC m=+0.170700219 container create 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-10T06:39:57.572 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 podman[33443]: 2024-02-10 06:39:57.571778644 +0000 UTC m=+0.389968906 container init 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-02-10T06:39:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 podman[33443]: 2024-02-10 06:39:57.577510282 +0000 UTC m=+0.395700556 container start 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-10T06:39:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T06:39:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pidfile_write: ignore empty --pid-file 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: load: jerasure load: lrc 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: RocksDB version: 7.9.2 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Git sha 0 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: DB SUMMARY 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: DB Session ID: SFOULCAMC0SV5QGL7IZH 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: CURRENT file: CURRENT 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-10T06:39:57.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.error_if_exists: 0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.create_if_missing: 0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.paranoid_checks: 1 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.env: 0x557628db2ae0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.info_log: 0x55762a8f38c0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.statistics: (nil) 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.use_fsync: 0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_log_file_size: 0 2024-02-10T06:39:57.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T06:39:57.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T06:39:57.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T06:39:57.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T06:39:57.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.allow_fallocate: 1 2024-02-10T06:39:57.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T06:39:57.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T06:39:57.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.use_direct_reads: 0 2024-02-10T06:39:57.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T06:39:57.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.db_log_dir: 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.wal_dir: 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.write_buffer_manager: 0x55762a8c7400 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T06:39:57.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.unordered_write: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.row_cache: None 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.wal_filter: None 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.two_write_queues: 0 2024-02-10T06:39:57.879 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.wal_compression: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.atomic_flush: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.log_readahead_size: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_background_jobs: 2 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_background_compactions: -1 2024-02-10T06:39:57.880 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_subcompactions: 1 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_open_files: -1 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_background_flushes: -1 2024-02-10T06:39:57.881 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Compression algorithms supported: 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kZSTD supported: 0 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kXpressCompression supported: 0 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kZlibCompression supported: 1 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kSnappyCompression supported: 1 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kLZ4Compression supported: 1 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: kBZip2Compression supported: 0 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.merge_operator: 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_filter: None 2024-02-10T06:39:57.882 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55762a8f39c0) 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: cache_index_and_filter_blocks: 1 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: pin_top_level_index_and_filter: 1 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: index_type: 0 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: data_block_index_type: 0 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: index_shortening: 1 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: checksum: 4 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: no_block_cache: 0 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache: 0x55762a8f7090 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache_name: BinnedLRUCache 2024-02-10T06:39:57.883 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache_options: 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: capacity : 536870912 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: num_shard_bits : 4 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: strict_capacity_limit : 0 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: high_pri_pool_ratio: 0.000 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_cache_compressed: (nil) 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: persistent_cache: (nil) 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_size: 4096 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_size_deviation: 10 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_restart_interval: 16 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: index_block_restart_interval: 1 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: metadata_block_size: 4096 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: partition_filters: 0 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: use_delta_encoding: 1 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: filter_policy: bloomfilter 2024-02-10T06:39:57.884 INFO:journalctl@ceph.mon.b.smithi184.stdout: whole_key_filtering: 1 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: verify_compression: 0 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: read_amp_bytes_per_bit: 0 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: format_version: 5 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: enable_index_compression: 1 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: block_align: 0 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: max_auto_readahead_size: 262144 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: prepopulate_block_cache: 0 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: initial_auto_readahead_size: 8192 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression: NoCompression 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T06:39:57.885 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.num_levels: 7 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T06:39:57.886 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T06:39:57.887 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T06:39:57.888 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.table_properties_collectors: 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.inplace_update_support: 0 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T06:39:57.889 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.bloom_locality: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.max_successive_merges: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.ttl: 2592000 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.enable_blob_files: false 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.min_blob_size: 0 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T06:39:57.890 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9c63fa8c-5290-448f-b91b-378b4786e634 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547197607783, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547197608327, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707547197, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9c63fa8c-5290-448f-b91b-378b4786e634", "db_session_id": "SFOULCAMC0SV5QGL7IZH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547197608424, "job": 1, "event": "recovery_finished"} 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55762a9e8000 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: DB pointer 0x55762a9d2000 2024-02-10T06:39:57.891 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** DB Stats ** 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** Compaction Stats [default] ** 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:39:57.892 INFO:journalctl@ceph.mon.b.smithi184.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** Compaction Stats [default] ** 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T06:39:57.893 INFO:journalctl@ceph.mon.b.smithi184.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: Block cache BinnedLRUCache@0x55762a8f7090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: using public_addr v2:172.21.15.184:0/0 -> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: starting mon.b rank -1 at public addrs [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] at bind addrs [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(???) e0 preinit fsid df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).mds e1 new map 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: e1 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T06:39:57.894 INFO:journalctl@ceph.mon.b.smithi184.stdout: legacy client fscid: -1 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout: 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout: No filesystems configured 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Added host smithi184 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.895 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/661684146' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/661684146' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:39:57.896 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:39:57.897 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Saving service mon spec with placement smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b;count:3 2024-02-10T06:39:57.898 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: Deploying daemon mon.b on smithi184 2024-02-10T06:39:57.898 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.898 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.898 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/3646760626' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:39:57.898 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 ceph-mon[33513]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T06:39:57.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:57 smithi174 ceph-mon[29878]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:39:57.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:39:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/3646760626' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:39:58.275 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 bash[33443]: 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe 2024-02-10T06:39:58.275 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:39:57 smithi184 systemd[1]: Started Ceph mon.b for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:39:59.302 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T06:39:59.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T06:39:59.562 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:40:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:40:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: mon.a calling monitor election 2024-02-10T06:40:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: mon.b calling monitor election 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: monmap e2: 2 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: fsmap 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: mgrmap e13: y(active, since 40s) 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: overall HEALTH_OK 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:02.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:02.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:03.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:40:03.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: mon.a calling monitor election 2024-02-10T06:40:03.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:03.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:03.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: mon.b calling monitor election 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: monmap e2: 2 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: fsmap 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: mgrmap e13: y(active, since 40s) 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: overall HEALTH_OK 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:03.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:04.089 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:03 smithi174 ceph-mon[29878]: Deploying daemon mon.c on smithi174 2024-02-10T06:40:04.089 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:04.120 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.118+0000 7fa60b60b700 1 -- 172.21.15.184:0/2540808965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5f00035c0 msgr2=0x7fa5f0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:04.121 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.118+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2540808965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5f00035c0 0x7fa5f0005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa604069fe0 tx=0x7fa60000feb0 comp rx=0 tx=0).stop 2024-02-10T06:40:04.121 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.119+0000 7fa60b60b700 1 -- 172.21.15.184:0/2540808965 shutdown_connections 2024-02-10T06:40:04.121 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.119+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2540808965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5f00035c0 0x7fa5f0005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:04.121 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.119+0000 7fa60b60b700 1 -- 172.21.15.184:0/2540808965 >> 172.21.15.184:0/2540808965 conn(0x7fa6040fc220 msgr2=0x7fa6040fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:04.121 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.119+0000 7fa60b60b700 1 -- 172.21.15.184:0/2540808965 shutdown_connections 2024-02-10T06:40:04.121 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.119+0000 7fa60b60b700 1 -- 172.21.15.184:0/2540808965 wait complete. 2024-02-10T06:40:04.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:03 smithi184 ceph-mon[33513]: Deploying daemon mon.c on smithi174 2024-02-10T06:40:04.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa60b60b700 1 Processor -- start 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa60b60b700 1 -- start start 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa60b60b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa60b60b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa60419d470 0x7fa6041a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa60b60b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60419da00 con 0x7fa60419d470 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa60b60b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60419db70 con 0x7fa5f00035c0 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa6093a7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa6093a7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:36100/0 (socket says 172.21.15.184:36100) 2024-02-10T06:40:04.123 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa6093a7700 1 -- 172.21.15.184:0/2413005523 learned_addr learned my addr 172.21.15.184:0/2413005523 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.120+0000 7fa6093a7700 1 -- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 msgr2=0x7fa60419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa6093a7700 1 -- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 msgr2=0x7fa60419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa6093a7700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa608ba6700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa60419d470 0x7fa6041a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa6093a7700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa608ba6700 1 -- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 msgr2=0x7fa60419cf30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa608ba6700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa608ba6700 1 -- 172.21.15.184:0/2413005523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60000f900 con 0x7fa60419d470 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa608ba6700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa60419d470 0x7fa6041a1800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400b4d0 tx=0x7fa5f400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa5f4011630 con 0x7fa60419d470 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa5f4011c40 con 0x7fa60419d470 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6041a1da0 con 0x7fa60419d470 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa5f4010c90 con 0x7fa60419d470 2024-02-10T06:40:04.124 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.121+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6041a22f0 con 0x7fa60419d470 2024-02-10T06:40:04.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.122+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fa5f401a400 con 0x7fa60419d470 2024-02-10T06:40:04.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.123+0000 7fa5fa7fc700 1 --2- 172.21.15.184:0/2413005523 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa5e8038d30 0x7fa5e803b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:04.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.123+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa604196c80 con 0x7fa60419d470 2024-02-10T06:40:04.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.123+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fa5f404b850 con 0x7fa60419d470 2024-02-10T06:40:04.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.123+0000 7fa6093a7700 1 --2- 172.21.15.184:0/2413005523 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa5e8038d30 0x7fa5e803b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:04.126 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.123+0000 7fa6093a7700 1 --2- 172.21.15.184:0/2413005523 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa5e8038d30 0x7fa5e803b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa600001eb0 tx=0x7fa600005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:04.129 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.128+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5f401acb0 con 0x7fa60419d470 2024-02-10T06:40:04.405 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.404+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa604063350 con 0x7fa60419d470 2024-02-10T06:40:04.405 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.404+0000 7fa5fa7fc700 1 -- 172.21.15.184:0/2413005523 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7fa5f40172c0 con 0x7fa60419d470 2024-02-10T06:40:04.406 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:40:04.406 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"df845a14-c7de-11ee-95b8-87774f69a715","modified":"2024-02-10T06:39:57.630692Z","created":"2024-02-10T06:38:12.344337Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T06:40:04.407 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa5e8038d30 msgr2=0x7fa5e803b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2413005523 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa5e8038d30 0x7fa5e803b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa600001eb0 tx=0x7fa600005a90 comp rx=0 tx=0).stop 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa60419d470 msgr2=0x7fa6041a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa60419d470 0x7fa6041a1800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400b4d0 tx=0x7fa5f400b890 comp rx=0 tx=0).stop 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 shutdown_connections 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2413005523 >> v2:172.21.15.174:6800/2183136494 conn(0x7fa5e8038d30 0x7fa5e803b1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f00035c0 0x7fa60419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 --2- 172.21.15.184:0/2413005523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa60419d470 0x7fa6041a1800 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 >> 172.21.15.184:0/2413005523 conn(0x7fa6040fc220 msgr2=0x7fa6040fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.406+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 shutdown_connections 2024-02-10T06:40:04.408 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:04.407+0000 7fa60b60b700 1 -- 172.21.15.184:0/2413005523 wait complete. 2024-02-10T06:40:04.409 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-02-10T06:40:04.987 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:04 smithi184 ceph-mon[33513]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:04.987 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/2413005523' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:04.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:40:04 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:40:04.630+0000 7f2b7f7d8700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-10T06:40:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:04 smithi174 ceph-mon[29878]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/2413005523' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:06.013 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T06:40:06.014 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T06:40:06.267 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:40:06.355 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 podman[39485]: 2024-02-10 06:40:06.23940452 +0000 UTC m=+0.278216018 container create 564f3843f9f0df60015ab92b343714c73f3db2b0f74a87e207568cda93fd934a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:40:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 podman[39485]: 2024-02-10 06:40:06.442014542 +0000 UTC m=+0.480826059 container init 564f3843f9f0df60015ab92b343714c73f3db2b0f74a87e207568cda93fd934a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:40:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 podman[39485]: 2024-02-10 06:40:06.448071245 +0000 UTC m=+0.486882746 container start 564f3843f9f0df60015ab92b343714c73f3db2b0f74a87e207568cda93fd934a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T06:40:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T06:40:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T06:40:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pidfile_write: ignore empty --pid-file 2024-02-10T06:40:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: load: jerasure load: lrc 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: RocksDB version: 7.9.2 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Git sha 0 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: DB SUMMARY 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: DB Session ID: 7ZY0SKGS044XJREPNE5L 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: CURRENT file: CURRENT 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.error_if_exists: 0 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.create_if_missing: 0 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.paranoid_checks: 1 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T06:40:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T06:40:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T06:40:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.env: 0x559ec43fcae0 2024-02-10T06:40:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T06:40:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.info_log: 0x559ec58b18c0 2024-02-10T06:40:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T06:40:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.statistics: (nil) 2024-02-10T06:40:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.use_fsync: 0 2024-02-10T06:40:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_log_file_size: 0 2024-02-10T06:40:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T06:40:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.allow_fallocate: 1 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.use_direct_reads: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.db_log_dir: 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.wal_dir: 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T06:40:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.write_buffer_manager: 0x559ec5885400 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.unordered_write: 0 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T06:40:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.row_cache: None 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.wal_filter: None 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.two_write_queues: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.wal_compression: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.atomic_flush: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.log_readahead_size: 0 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T06:40:06.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_background_jobs: 2 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_background_compactions: -1 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_subcompactions: 1 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T06:40:06.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_open_files: -1 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_background_flushes: -1 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Compression algorithms supported: 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kZSTD supported: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kXpressCompression supported: 0 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kZlibCompression supported: 1 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kSnappyCompression supported: 1 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kLZ4Compression supported: 1 2024-02-10T06:40:06.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: kBZip2Compression supported: 0 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.merge_operator: 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_filter: None 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559ec58b19c0) 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T06:40:06.748 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x559ec58b5090 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-02-10T06:40:06.749 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-02-10T06:40:06.750 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression: NoCompression 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.num_levels: 7 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T06:40:06.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T06:40:06.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T06:40:06.753 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.table_properties_collectors: 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.inplace_update_support: 0 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T06:40:06.754 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.bloom_locality: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.max_successive_merges: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.ttl: 2592000 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.enable_blob_files: false 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.min_blob_size: 0 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T06:40:06.755 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2cafbd5a-f6ba-4a17-b536-8c31194e20a9 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547206478169, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-10T06:40:06.756 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547206478553, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707547206, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2cafbd5a-f6ba-4a17-b536-8c31194e20a9", "db_session_id": "7ZY0SKGS044XJREPNE5L", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707547206478633, "job": 1, "event": "recovery_finished"} 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559ec59a6000 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: DB pointer 0x559ec5990000 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T06:40:06.757 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-02-10T06:40:06.758 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x559ec58b5090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: using public_addrv [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] at bind addrs [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(???) e0 preinit fsid df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:40:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).mds e1 new map 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Added host smithi184 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/661684146' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/661684146' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:40:06.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:06.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Saving service mon spec with placement smithi174:172.21.15.174=a;smithi174:[v2:172.21.15.174:3301,v1:172.21.15.174:6790]=c;smithi184:172.21.15.184=b;count:3 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Deploying daemon mon.b on smithi184 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/3646760626' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.a calling monitor election 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.b calling monitor election 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: monmap e2: 2 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: fsmap 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mgrmap e13: y(active, since 40s) 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: overall HEALTH_OK 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: Deploying daemon mon.c on smithi174 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/2413005523' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:06.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 ceph-mon[39523]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T06:40:06.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 bash[39485]: 564f3843f9f0df60015ab92b343714c73f3db2b0f74a87e207568cda93fd934a 2024-02-10T06:40:06.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:06 smithi174 systemd[1]: Started Ceph mon.c for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:40:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:40:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: mon.a calling monitor election 2024-02-10T06:40:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: mon.b calling monitor election 2024-02-10T06:40:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: mon.c calling monitor election 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: monmap e3: 3 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: fsmap 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: mgrmap e13: y(active, since 49s) 2024-02-10T06:40:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: overall HEALTH_OK 2024-02-10T06:40:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:40:11.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: mon.a calling monitor election 2024-02-10T06:40:11.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: mon.b calling monitor election 2024-02-10T06:40:11.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: mon.c calling monitor election 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: monmap e3: 3 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: fsmap 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:11.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: mgrmap e13: y(active, since 49s) 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: overall HEALTH_OK 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: mon.a calling monitor election 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: mon.b calling monitor election 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:40:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: mon.c calling monitor election 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: monmap e3: 3 mons at {a=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],b=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0],c=[v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: fsmap 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: mgrmap e13: y(active, since 49s) 2024-02-10T06:40:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: overall HEALTH_OK 2024-02-10T06:40:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:12.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 -- 172.21.15.184:0/3577713802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5ac0035c0 msgr2=0x7fd5ac005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:12.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/3577713802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5ac0035c0 0x7fd5ac005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0101730 tx=0x7fd5b8009e80 comp rx=0 tx=0).stop 2024-02-10T06:40:12.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 -- 172.21.15.184:0/3577713802 shutdown_connections 2024-02-10T06:40:12.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/3577713802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5ac0035c0 0x7fd5ac005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 -- 172.21.15.184:0/3577713802 >> 172.21.15.184:0/3577713802 conn(0x7fd5c00fd310 msgr2=0x7fd5c00ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:12.258 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 -- 172.21.15.184:0/3577713802 shutdown_connections 2024-02-10T06:40:12.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.256+0000 7fd5c60ec700 1 -- 172.21.15.184:0/3577713802 wait complete. 2024-02-10T06:40:12.259 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.257+0000 7fd5c60ec700 1 Processor -- start 2024-02-10T06:40:12.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 -- start start 2024-02-10T06:40:12.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:12.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 0x7fd5c01967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:12.260 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c0196d30 0x7fd5c01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c019d1d0 con 0x7fd5c0196d30 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01a3480 con 0x7fd5ac0035c0 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5c60ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01a35f0 con 0x7fd5c019cb60 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c0196d30 0x7fd5c01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41832/0 (socket says 172.21.15.184:41832) 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 -- 172.21.15.184:0/2028570760 learned_addr learned my addr 172.21.15.184:0/2028570760 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5beffd700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 0x7fd5c01967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 msgr2=0x7fd5c019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 msgr2=0x7fd5c019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5beffd700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 msgr2=0x7fd5c01967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5beffd700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 msgr2=0x7fd5c01967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5beffd700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 0x7fd5c01967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-10T06:40:12.261 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5beffd700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 0x7fd5c01967f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bf7fe700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bffff700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 msgr2=0x7fd5c01967f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bffff700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 0x7fd5c01967f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bffff700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 msgr2=0x7fd5c019c620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bffff700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.258+0000 7fd5bffff700 1 -- 172.21.15.184:0/2028570760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b80098d0 con 0x7fd5c0196d30 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.259+0000 7fd5bffff700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c0196d30 0x7fd5c01971e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0006fd0 tx=0x7fd5b000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.259+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0011830 con 0x7fd5c0196d30 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.259+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd5b0011e40 con 0x7fd5c0196d30 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.259+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0010e30 con 0x7fd5c0196d30 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.259+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c0073fb0 con 0x7fd5c0196d30 2024-02-10T06:40:12.262 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.259+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c0074500 con 0x7fd5c0196d30 2024-02-10T06:40:12.263 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.260+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c01905b0 con 0x7fd5c0196d30 2024-02-10T06:40:12.263 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.260+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fd5b001a400 con 0x7fd5c0196d30 2024-02-10T06:40:12.263 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.260+0000 7fd5bcff9700 1 --2- 172.21.15.184:0/2028570760 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd5a00393d0 0x7fd5a003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:12.263 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.260+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd5b004b720 con 0x7fd5c0196d30 2024-02-10T06:40:12.263 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.260+0000 7fd5bf7fe700 1 --2- 172.21.15.184:0/2028570760 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd5a00393d0 0x7fd5a003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:12.263 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.261+0000 7fd5bf7fe700 1 --2- 172.21.15.184:0/2028570760 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd5a00393d0 0x7fd5a003b890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009160 tx=0x7fd5b80090f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:12.268 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.267+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5b00178b0 con 0x7fd5c0196d30 2024-02-10T06:40:12.575 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.574+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd5c0102f40 con 0x7fd5c0196d30 2024-02-10T06:40:12.576 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.575+0000 7fd5bcff9700 1 -- 172.21.15.184:0/2028570760 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1289 (secure 0 0 0) 0x7fd5b00172c0 con 0x7fd5c0196d30 2024-02-10T06:40:12.576 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-02-10T06:40:12.576 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"df845a14-c7de-11ee-95b8-87774f69a715","modified":"2024-02-10T06:40:06.498752Z","created":"2024-02-10T06:38:12.344337Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3301","nonce":0},{"type":"v1","addr":"172.21.15.174:6790","nonce":0}]},"addr":"172.21.15.174:6790/0","public_addr":"172.21.15.174:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd5a00393d0 msgr2=0x7fd5a003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/2028570760 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd5a00393d0 0x7fd5a003b890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009160 tx=0x7fd5b80090f0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c0196d30 msgr2=0x7fd5c01971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c0196d30 0x7fd5c01971e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0006fd0 tx=0x7fd5b000b7e0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 shutdown_connections 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c0196d30 0x7fd5c01971e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/2028570760 >> v2:172.21.15.174:6800/2183136494 conn(0x7fd5a00393d0 0x7fd5a003b890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5ac0035c0 0x7fd5c019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 --2- 172.21.15.184:0/2028570760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5c019cb60 0x7fd5c01967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 >> 172.21.15.184:0/2028570760 conn(0x7fd5c00fd310 msgr2=0x7fd5c00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:12.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 shutdown_connections 2024-02-10T06:40:12.580 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:12.578+0000 7fd5c60ec700 1 -- 172.21.15.184:0/2028570760 wait complete. 2024-02-10T06:40:12.580 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-02-10T06:40:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:12 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:12 smithi184 ceph-mon[33513]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:12 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:12 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/2028570760' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[29878]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/2028570760' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[39523]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:40:12.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/2028570760' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T06:40:13.608 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-10T06:40:13.608 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph config generate-minimal-conf 2024-02-10T06:40:13.856 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:40:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:14.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:40:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:14.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:15.227 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:15 smithi174 ceph-mon[29878]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T06:40:15.227 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:15 smithi174 ceph-mon[29878]: Reconfiguring daemon mon.a on smithi174 2024-02-10T06:40:15.227 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:15 smithi174 ceph-mon[29878]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:15.227 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:15 smithi174 ceph-mon[39523]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T06:40:15.227 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:15 smithi174 ceph-mon[39523]: Reconfiguring daemon mon.a on smithi174 2024-02-10T06:40:15.228 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:15 smithi174 ceph-mon[39523]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:15 smithi184 ceph-mon[33513]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T06:40:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:15 smithi184 ceph-mon[33513]: Reconfiguring daemon mon.a on smithi174 2024-02-10T06:40:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:15 smithi184 ceph-mon[33513]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 -- 172.21.15.174:0/631380820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 msgr2=0x7fe1b0106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/631380820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b0106a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009910 tx=0x7fe1a0009c20 comp rx=0 tx=0).stop 2024-02-10T06:40:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 -- 172.21.15.174:0/631380820 shutdown_connections 2024-02-10T06:40:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/631380820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b0106a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 -- 172.21.15.174:0/631380820 >> 172.21.15.174:0/631380820 conn(0x7fe1b00fc110 msgr2=0x7fe1b00fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 -- 172.21.15.174:0/631380820 shutdown_connections 2024-02-10T06:40:16.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.111+0000 7fe1b5f23700 1 -- 172.21.15.174:0/631380820 wait complete. 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 Processor -- start 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 -- start start 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1b019cb90 0x7fe1b01967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe1b0196d10 0x7fe1b01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b019d200 con 0x7fe1b0106670 2024-02-10T06:40:16.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01a34b0 con 0x7fe1b019cb90 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1b5f23700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01a3620 con 0x7fe1b0196d10 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1af7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1af7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47660/0 (socket says 172.21.15.174:47660) 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1af7fe700 1 -- 172.21.15.174:0/3558484163 learned_addr learned my addr 172.21.15.174:0/3558484163 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.112+0000 7fe1affff700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe1b0196d10 0x7fe1b01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1af7fe700 1 -- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe1b0196d10 msgr2=0x7fe1b01971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1af7fe700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe1b0196d10 0x7fe1b01971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1af7fe700 1 -- 172.21.15.174:0/3558484163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1b019cb90 msgr2=0x7fe1b01967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1af7fe700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1b019cb90 0x7fe1b01967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1af7fe700 1 -- 172.21.15.174:0/3558484163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00095f0 con 0x7fe1b0106670 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1af7fe700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b019c650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000b5c0 tx=0x7fe1a000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a001d070 con 0x7fe1b0106670 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe1a00050d0 con 0x7fe1b0106670 2024-02-10T06:40:16.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b00fc4f0 con 0x7fe1b0106670 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000fc40 con 0x7fe1b0106670 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.113+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b00fcac0 con 0x7fe1b0106670 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.114+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7fe1a002b440 con 0x7fe1b0106670 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.114+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b0101d20 con 0x7fe1b0106670 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.114+0000 7fe1acff9700 1 --2- 172.21.15.174:0/3558484163 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1980393d0 0x7fe19803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.114+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe1a004cca0 con 0x7fe1b0106670 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.114+0000 7fe1aeffd700 1 --2- 172.21.15.174:0/3558484163 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1980393d0 0x7fe19803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:16.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.115+0000 7fe1aeffd700 1 --2- 172.21.15.174:0/3558484163 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1980393d0 0x7fe19803b890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40098e0 tx=0x7fe1a4006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:16.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.119+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1a0020960 con 0x7fe1b0106670 2024-02-10T06:40:16.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.315+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe1b0197da0 con 0x7fe1b0106670 2024-02-10T06:40:16.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.315+0000 7fe1acff9700 1 -- 172.21.15.174:0/3558484163 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7fe1a0020370 con 0x7fe1b0106670 2024-02-10T06:40:16.316 INFO:teuthology.orchestra.run.smithi174.stdout:# minimal ceph.conf for df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:40:16.316 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2024-02-10T06:40:16.316 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:40:16.317 INFO:teuthology.orchestra.run.smithi174.stdout: mon_host = [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1980393d0 msgr2=0x7fe19803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/3558484163 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1980393d0 0x7fe19803b890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40098e0 tx=0x7fe1a4006dd0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 msgr2=0x7fe1b019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b019c650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe1a000b5c0 tx=0x7fe1a000bd80 comp rx=0 tx=0).stop 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 shutdown_connections 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe1b0196d10 0x7fe1b01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b0106670 0x7fe1b019c650 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/3558484163 >> v2:172.21.15.174:6800/2183136494 conn(0x7fe1980393d0 0x7fe19803b890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 --2- 172.21.15.174:0/3558484163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1b019cb90 0x7fe1b01967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 >> 172.21.15.174:0/3558484163 conn(0x7fe1b00fc110 msgr2=0x7fe1b01041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 shutdown_connections 2024-02-10T06:40:16.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:16.318+0000 7fe1b5f23700 1 -- 172.21.15.174:0/3558484163 wait complete. 2024-02-10T06:40:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3558484163' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3558484163' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3558484163' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:17.115 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-10T06:40:17.115 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:40:17.115 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T06:40:17.146 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:40:17.147 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:40:17.214 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:40:17.214 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T06:40:17.248 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:40:17.248 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:40:17.319 INFO:tasks.cephadm:Adding mgr.y on smithi174 2024-02-10T06:40:17.319 INFO:tasks.cephadm:Adding mgr.x on smithi184 2024-02-10T06:40:17.319 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply mgr '2;smithi174=y;smithi184=x' 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[39523]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[29878]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:17.400 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:17.401 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:17.401 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:17.401 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:17.592 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:40:17.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:17 smithi184 ceph-mon[33513]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: Reconfiguring mon.c (monmap changed)... 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: Reconfiguring daemon mon.c on smithi174 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:18.678 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: Reconfiguring mon.c (monmap changed)... 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: Reconfiguring daemon mon.c on smithi174 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: Reconfiguring mon.c (monmap changed)... 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: Reconfiguring daemon mon.c on smithi174 2024-02-10T06:40:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T06:40:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T06:40:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 -- 172.21.15.184:0/2830539540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8100590 msgr2=0x7f79c81009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2830539540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8100590 0x7f79c81009b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00b0a0 tx=0x7f79bc00b3b0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 -- 172.21.15.184:0/2830539540 shutdown_connections 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2830539540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8100590 0x7f79c81009b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 -- 172.21.15.184:0/2830539540 >> 172.21.15.184:0/2830539540 conn(0x7f79c80fbb30 msgr2=0x7f79c80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 -- 172.21.15.184:0/2830539540 shutdown_connections 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.385+0000 7f79ced9e700 1 -- 172.21.15.184:0/2830539540 wait complete. 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 Processor -- start 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 -- start start 2024-02-10T06:40:19.389 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 0x7f79c806aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8199080 0x7f79c819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79c819ba50 0x7f79c819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819e680 con 0x7f79c8199080 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819e7f0 con 0x7f79c8100590 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.386+0000 7f79ced9e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819e9b0 con 0x7f79c819ba50 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 0x7f79c806aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 0x7f79c806aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41838/0 (socket says 172.21.15.184:41838) 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 -- 172.21.15.184:0/2330148291 learned_addr learned my addr 172.21.15.184:0/2330148291 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cd59b700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8199080 0x7f79c819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79ce59d700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79c819ba50 0x7f79c819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 -- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79c819ba50 msgr2=0x7f79c819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79c819ba50 0x7f79c819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.390 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 -- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8199080 msgr2=0x7f79c819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8199080 0x7f79c819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 -- 172.21.15.184:0/2330148291 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc009d00 con 0x7f79c8100590 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79ce59d700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79c819ba50 0x7f79c819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79cdd9c700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 0x7f79c806aac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00d010 tx=0x7f79bc007910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc00e040 con 0x7f79c8100590 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c819ec30 con 0x7f79c8100590 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f79bc005520 con 0x7f79c8100590 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0127b0 con 0x7f79c8100590 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.387+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c819f180 con 0x7f79c8100590 2024-02-10T06:40:19.391 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.388+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f79bc019070 con 0x7f79c8100590 2024-02-10T06:40:19.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.388+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c818f0a0 con 0x7f79c8100590 2024-02-10T06:40:19.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.388+0000 7f79c6ffd700 1 --2- 172.21.15.184:0/2330148291 >> v2:172.21.15.174:6800/2183136494 conn(0x7f79b80393d0 0x7f79b803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:19.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.388+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f79bc04cd20 con 0x7f79c8100590 2024-02-10T06:40:19.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.388+0000 7f79cd59b700 1 --2- 172.21.15.184:0/2330148291 >> v2:172.21.15.174:6800/2183136494 conn(0x7f79b80393d0 0x7f79b803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:19.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.389+0000 7f79cd59b700 1 --2- 172.21.15.184:0/2330148291 >> v2:172.21.15.174:6800/2183136494 conn(0x7f79b80393d0 0x7f79b803b890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009ad0 tx=0x7f79b4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:19.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.394+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79bc0196e0 con 0x7f79c8100590 2024-02-10T06:40:19.605 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.604+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi174=y;smithi184=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c8062160 con 0x7f79b80393d0 2024-02-10T06:40:19.606 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:19 smithi184 ceph-mon[33513]: Reconfiguring mon.b (monmap changed)... 2024-02-10T06:40:19.606 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:19 smithi184 ceph-mon[33513]: Reconfiguring daemon mon.b on smithi184 2024-02-10T06:40:19.606 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:19 smithi184 ceph-mon[33513]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:19.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.611+0000 7f79c6ffd700 1 -- 172.21.15.184:0/2330148291 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f79c8062160 con 0x7f79b80393d0 2024-02-10T06:40:19.612 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-02-10T06:40:19.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 >> v2:172.21.15.174:6800/2183136494 conn(0x7f79b80393d0 msgr2=0x7f79b803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:19.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2330148291 >> v2:172.21.15.174:6800/2183136494 conn(0x7f79b80393d0 0x7f79b803b890 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009ad0 tx=0x7f79b4009210 comp rx=0 tx=0).stop 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 msgr2=0x7f79c806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 0x7f79c806aac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00d010 tx=0x7f79bc007910 comp rx=0 tx=0).stop 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 shutdown_connections 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79c819ba50 0x7f79c819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2330148291 >> v2:172.21.15.174:6800/2183136494 conn(0x7f79b80393d0 0x7f79b803b890 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c8100590 0x7f79c806aac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 --2- 172.21.15.184:0/2330148291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c8199080 0x7f79c819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 >> 172.21.15.184:0/2330148291 conn(0x7f79c80fbb30 msgr2=0x7f79c80fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 shutdown_connections 2024-02-10T06:40:19.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:40:19.613+0000 7f79ced9e700 1 -- 172.21.15.184:0/2330148291 wait complete. 2024-02-10T06:40:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:19 smithi174 ceph-mon[29878]: Reconfiguring mon.b (monmap changed)... 2024-02-10T06:40:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:19 smithi174 ceph-mon[29878]: Reconfiguring daemon mon.b on smithi184 2024-02-10T06:40:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:19 smithi174 ceph-mon[29878]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:19 smithi174 ceph-mon[39523]: Reconfiguring mon.b (monmap changed)... 2024-02-10T06:40:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:19 smithi174 ceph-mon[39523]: Reconfiguring daemon mon.b on smithi184 2024-02-10T06:40:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:19 smithi174 ceph-mon[39523]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:20.783 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi174=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:20.783 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: Saving service mgr spec with placement smithi174=y;smithi184=x;count:2 2024-02-10T06:40:20.783 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.783 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:20.784 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:20 smithi184 ceph-mon[33513]: Deploying daemon mgr.x on smithi184 2024-02-10T06:40:20.811 DEBUG:teuthology.orchestra.run.smithi184:mgr.x> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.x.service 2024-02-10T06:40:20.813 INFO:tasks.cephadm:Deploying OSDs... 2024-02-10T06:40:20.813 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:40:20.813 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:40:20.833 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T06:40:20.833 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-02-10T06:40:20.891 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-02-10 06:40:05.793408380 +0000 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-02-10 06:34:36.501556703 +0000 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-02-10 06:34:36.501556703 +0000 2024-02-10T06:40:20.892 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-02-10 06:34:36.501556703 +0000 2024-02-10T06:40:20.892 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi174=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: Saving service mgr spec with placement smithi174=y;smithi184=x;count:2 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:20.960 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[29878]: Deploying daemon mgr.x on smithi184 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi174=y;smithi184=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: Saving service mgr spec with placement smithi174=y;smithi184=x;count:2 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:40:20.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:20.962 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:20 smithi174 ceph-mon[39523]: Deploying daemon mgr.x on smithi184 2024-02-10T06:40:20.962 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-02-10T06:40:20.963 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-02-10T06:40:20.963 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000284571 s, 1.8 MB/s 2024-02-10T06:40:20.964 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T06:40:21.022 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-02-10T06:40:21.079 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-02-10 06:40:05.794408366 +0000 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-02-10 06:34:36.846552110 +0000 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-02-10 06:34:36.846552110 +0000 2024-02-10T06:40:21.080 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-02-10 06:34:36.846552110 +0000 2024-02-10T06:40:21.080 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T06:40:21.148 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-02-10T06:40:21.148 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-02-10T06:40:21.148 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000203182 s, 2.5 MB/s 2024-02-10T06:40:21.149 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T06:40:21.206 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-02-10 06:40:05.794408366 +0000 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-02-10 06:34:37.198547424 +0000 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-02-10 06:34:37.198547424 +0000 2024-02-10T06:40:21.262 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-02-10 06:34:37.198547424 +0000 2024-02-10T06:40:21.263 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T06:40:21.329 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-02-10T06:40:21.329 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-02-10T06:40:21.329 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000210102 s, 2.4 MB/s 2024-02-10T06:40:21.330 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T06:40:21.386 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-02-10 06:40:05.795408354 +0000 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-02-10 06:34:37.546542791 +0000 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-02-10 06:34:37.546542791 +0000 2024-02-10T06:40:21.442 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-02-10 06:34:37.546542791 +0000 2024-02-10T06:40:21.443 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T06:40:21.507 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-02-10T06:40:21.507 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-02-10T06:40:21.507 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000201282 s, 2.5 MB/s 2024-02-10T06:40:21.508 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T06:40:21.564 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:40:21.564 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T06:40:21.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T06:40:21.580 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-10 06:39:56.457436011 +0000 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-10 06:34:44.588470183 +0000 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-10 06:34:44.588470183 +0000 2024-02-10T06:40:21.635 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-10 06:34:44.588470183 +0000 2024-02-10T06:40:21.635 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T06:40:21.701 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-10T06:40:21.701 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-10T06:40:21.701 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00024706 s, 2.1 MB/s 2024-02-10T06:40:21.703 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T06:40:21.759 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-10 06:39:56.457436011 +0000 2024-02-10T06:40:21.813 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-10 06:34:44.923463142 +0000 2024-02-10T06:40:21.814 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-10 06:34:44.924463121 +0000 2024-02-10T06:40:21.814 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-10 06:34:44.923463142 +0000 2024-02-10T06:40:21.814 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T06:40:21.879 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-10T06:40:21.879 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-10T06:40:21.879 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000216276 s, 2.4 MB/s 2024-02-10T06:40:21.880 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T06:40:21.936 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-02-10T06:40:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T06:40:21.991 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:21.991 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-10T06:40:21.991 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:21.992 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:21.992 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-10 06:39:56.457436011 +0000 2024-02-10T06:40:21.992 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-10 06:34:45.266455933 +0000 2024-02-10T06:40:21.992 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-10 06:34:45.266455933 +0000 2024-02-10T06:40:21.992 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-10 06:34:45.266455933 +0000 2024-02-10T06:40:21.992 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T06:40:22.057 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-10T06:40:22.057 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-10T06:40:22.057 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000205796 s, 2.5 MB/s 2024-02-10T06:40:22.058 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T06:40:22.114 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-02-10T06:40:22.169 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T06:40:22.169 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-02-10 06:39:56.458435992 +0000 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-02-10 06:34:45.612448661 +0000 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-02-10 06:34:45.612448661 +0000 2024-02-10T06:40:22.170 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-02-10 06:34:45.612448661 +0000 2024-02-10T06:40:22.170 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T06:40:22.236 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-02-10T06:40:22.236 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-02-10T06:40:22.236 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00020496 s, 2.5 MB/s 2024-02-10T06:40:22.237 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T06:40:22.300 INFO:tasks.cephadm:Deploying osd.0 on smithi174 with /dev/vg_nvme/lv_4... 2024-02-10T06:40:22.300 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-10T06:40:22.539 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:40:22.806 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:22 smithi184 systemd[1]: Starting Ceph mgr.x for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:40:23.058 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:22 smithi184 podman[35588]: 2024-02-10 06:40:22.708850928 +0000 UTC m=+0.017570480 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:40:23.059 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:22 smithi184 podman[35588]: 2024-02-10 06:40:22.808870586 +0000 UTC m=+0.117590130 container create eaa1f932db7adec413f9efd581e7352dc86b40fff6532a780096b86caf28dcdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-02-10T06:40:23.059 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 podman[35588]: 2024-02-10 06:40:23.052997405 +0000 UTC m=+0.361716948 container init eaa1f932db7adec413f9efd581e7352dc86b40fff6532a780096b86caf28dcdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-10T06:40:23.059 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 podman[35588]: 2024-02-10 06:40:23.058516553 +0000 UTC m=+0.367236100 container start eaa1f932db7adec413f9efd581e7352dc86b40fff6532a780096b86caf28dcdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-10T06:40:23.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mgr[35626]: -- 172.21.15.184:0/4283665291 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea8de101a0 con 0x55ea8dde6800 2024-02-10T06:40:23.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 bash[35588]: eaa1f932db7adec413f9efd581e7352dc86b40fff6532a780096b86caf28dcdb 2024-02-10T06:40:23.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 systemd[1]: Started Ceph mgr.x for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[29878]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[39523]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:23.651 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:23.719 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:23.621+0000 7f857cfb7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:40:23.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mon[33513]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:23.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:23.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:24.123 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:23 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:23.718+0000 7f857cfb7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:40:24.578 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-10T06:40:24.578 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-10T06:40:24.578 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-02-10T06:40:24.578 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-02-10T06:40:24.578 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121246 s, 865 MB/s 2024-02-10T06:40:24.578 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-02-10T06:40:24.749 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:24.404+0000 7f857cfb7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:40:24.749 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:24.498+0000 7f857cfb7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:40:24.749 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:24.648+0000 7f857cfb7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:40:24.749 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:24.748+0000 7f857cfb7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:40:25.090 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:24.830+0000 7f857cfb7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:40:25.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-02-10T06:40:25.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:25.089+0000 7f857cfb7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:40:25.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:25.198+0000 7f857cfb7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:40:25.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:25.283+0000 7f857cfb7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:40:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:25.379+0000 7f857cfb7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:25.555+0000 7f857cfb7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T06:40:25.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:26.289 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:26.529+0000 7f857cfb7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:26.660+0000 7f857cfb7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-mon[33513]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-mon[33513]: Reconfiguring daemon mgr.y on smithi174 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:26.662 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:26.674 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[29878]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T06:40:26.674 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[29878]: Reconfiguring daemon mgr.y on smithi174 2024-02-10T06:40:26.674 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:26.674 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:26.674 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:26.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[39523]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T06:40:26.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[39523]: Reconfiguring daemon mgr.y on smithi174 2024-02-10T06:40:26.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:26.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:26.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:26 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:27.284 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:26.996+0000 7f857cfb7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:40:27.285 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:27.086+0000 7f857cfb7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:40:27.285 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:27.283+0000 7f857cfb7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:40:27.443 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[29878]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:27.443 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:27.443 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[39523]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:27.444 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:27 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:27.623 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:27.470+0000 7f857cfb7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:40:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-mon[33513]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:27.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 -- 172.21.15.174:0/3350162815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4100ea0 msgr2=0x7f42b4101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 --2- 172.21.15.174:0/3350162815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4100ea0 0x7f42b4101320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009990 tx=0x7f42a8009ca0 comp rx=0 tx=0).stop 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 -- 172.21.15.174:0/3350162815 shutdown_connections 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 --2- 172.21.15.174:0/3350162815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b4101860 0x7f42b4109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 --2- 172.21.15.174:0/3350162815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42b41022d0 0x7f42b41026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 --2- 172.21.15.174:0/3350162815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4100ea0 0x7f42b4101320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 -- 172.21.15.174:0/3350162815 >> 172.21.15.174:0/3350162815 conn(0x7f42b40fb990 msgr2=0x7f42b40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 -- 172.21.15.174:0/3350162815 shutdown_connections 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.936+0000 7f42ba740700 1 -- 172.21.15.174:0/3350162815 wait complete. 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 Processor -- start 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 -- start start 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42b4100ea0 0x7f42b4104c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4101860 0x7f42b4103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b41022d0 0x7f42b41037d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4103f50 con 0x7f42b4101860 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b41040c0 con 0x7f42b4100ea0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42ba740700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b41a1480 con 0x7f42b41022d0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4101860 0x7f42b4103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b8cdd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b41022d0 0x7f42b41037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42b4100ea0 0x7f42b4104c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4101860 0x7f42b4103290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:51220/0 (socket says 172.21.15.174:51220) 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b37fe700 1 -- 172.21.15.174:0/4224268984 learned_addr learned my addr 172.21.15.174:0/4224268984 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b8cdd700 1 -- 172.21.15.174:0/4224268984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42b4100ea0 msgr2=0x7f42b4104c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b8cdd700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42b4100ea0 0x7f42b4104c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b8cdd700 1 -- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4101860 msgr2=0x7f42b4103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b8cdd700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4101860 0x7f42b4103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.937+0000 7f42b8cdd700 1 -- 172.21.15.174:0/4224268984 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80095f0 con 0x7f42b41022d0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.938+0000 7f42b8cdd700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b41022d0 0x7f42b41037d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00efc0 tx=0x7f42ac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:27.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.938+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac018070 con 0x7f42b41022d0 2024-02-10T06:40:27.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.938+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f42ac00f040 con 0x7f42b41022d0 2024-02-10T06:40:27.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.938+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00cc10 con 0x7f42b41022d0 2024-02-10T06:40:27.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.938+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b41a1700 con 0x7f42b41022d0 2024-02-10T06:40:27.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.938+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b41a1bb0 con 0x7f42b41022d0 2024-02-10T06:40:27.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.939+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 13) v1 ==== 45954+0+0 (secure 0 0 0) 0x7f42ac004000 con 0x7f42b41022d0 2024-02-10T06:40:27.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.939+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b4108a90 con 0x7f42b41022d0 2024-02-10T06:40:27.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.939+0000 7f42b17fa700 1 --2- 172.21.15.174:0/4224268984 >> v2:172.21.15.174:6800/2183136494 conn(0x7f429c039180 0x7f429c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:27.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.939+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f42ac04b720 con 0x7f42b41022d0 2024-02-10T06:40:27.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.939+0000 7f42b3fff700 1 --2- 172.21.15.174:0/4224268984 >> v2:172.21.15.174:6800/2183136494 conn(0x7f429c039180 0x7f429c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:27.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.939+0000 7f42b3fff700 1 --2- 172.21.15.174:0/4224268984 >> v2:172.21.15.174:6800/2183136494 conn(0x7f429c039180 0x7f429c03b640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009c60 tx=0x7f42a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:27.945 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:27.944+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42ac005000 con 0x7f42b41022d0 2024-02-10T06:40:28.017 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:27.751+0000 7f857cfb7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:40:28.017 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:28.016+0000 7f857cfb7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:40:28.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:28.146+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f42b410bf60 con 0x7f429c039180 2024-02-10T06:40:28.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:28.249+0000 7f857cfb7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:40:28.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[39523]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:28.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:40:28.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:40:28.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:29.000 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[39523]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:29.000 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[29878]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:29.000 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:40:29.000 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:40:29.000 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:29.000 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:28 smithi174 ceph-mon[29878]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:29.046 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-mon[33513]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:29.046 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:40:29.046 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:40:29.046 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:29.046 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:28 smithi184 ceph-mon[33513]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:29.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:29.045+0000 7f857cfb7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:40:29.872 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:29.614+0000 7f857cfb7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:40:29.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:29.709+0000 7f857cfb7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:40:29.994 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:29.993+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7f42ac01c3c0 con 0x7f42b41022d0 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[29878]: Standby manager daemon x started 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[39523]: Standby manager daemon x started 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:40:30.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:40:30.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:40:30.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:29 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:40:30.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:40:29.924+0000 7f857cfb7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:40:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-mon[33513]: Standby manager daemon x started 2024-02-10T06:40:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:40:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:40:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:40:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:29 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/771621974' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:40:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: mgrmap e14: y(active, since 67s), standbys: x 2024-02-10T06:40:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:40:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2020288432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]: dispatch 2024-02-10T06:40:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]: dispatch 2024-02-10T06:40:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]': finished 2024-02-10T06:40:31.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T06:40:31.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:31 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: mgrmap e14: y(active, since 67s), standbys: x 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2020288432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]: dispatch 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]: dispatch 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]': finished 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T06:40:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: mgrmap e14: y(active, since 67s), standbys: x 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2020288432' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]: dispatch 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]: dispatch 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b892cc18-df62-424f-8250-d301c822e8f1"}]': finished 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T06:40:31.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:31 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2660076103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:40:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2660076103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:40:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2660076103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:40:33.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:33 smithi184 ceph-mon[33513]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:33 smithi174 ceph-mon[29878]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:33 smithi174 ceph-mon[39523]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:35.700 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:35 smithi174 ceph-mon[29878]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:35.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:35 smithi174 ceph-mon[39523]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:35 smithi184 ceph-mon[33513]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:37 smithi174 ceph-mon[39523]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T06:40:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:37 smithi174 ceph-mon[29878]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T06:40:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:37.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:37 smithi184 ceph-mon[33513]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:37.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T06:40:37.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:38 smithi174 ceph-mon[29878]: Deploying daemon osd.0 on smithi174 2024-02-10T06:40:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:38 smithi174 ceph-mon[39523]: Deploying daemon osd.0 on smithi174 2024-02-10T06:40:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:38 smithi184 ceph-mon[33513]: Deploying daemon osd.0 on smithi174 2024-02-10T06:40:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:39 smithi174 ceph-mon[29878]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:39 smithi174 ceph-mon[39523]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:39 smithi184 ceph-mon[33513]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:41 smithi174 ceph-mon[29878]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:41 smithi174 ceph-mon[39523]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:41 smithi184 ceph-mon[33513]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:43.600 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[29878]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:43.600 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:43.601 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:43.601 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:43.601 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[39523]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:43.601 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:43.601 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:43.601 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:43 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:43 smithi184 ceph-mon[33513]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:43 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:43 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:43 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:45.697 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:45 smithi174 ceph-mon[29878]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:45.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:45 smithi174 ceph-mon[39523]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:45.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:45 smithi184 ceph-mon[33513]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:46.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.348+0000 7f42b17fa700 1 -- 172.21.15.174:0/4224268984 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f42b410bf60 con 0x7f429c039180 2024-02-10T06:40:46.351 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 0 on host 'smithi174' 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 >> v2:172.21.15.174:6800/2183136494 conn(0x7f429c039180 msgr2=0x7f429c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 --2- 172.21.15.174:0/4224268984 >> v2:172.21.15.174:6800/2183136494 conn(0x7f429c039180 0x7f429c03b640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009c60 tx=0x7f42a4009260 comp rx=0 tx=0).stop 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b41022d0 msgr2=0x7f42b41037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b41022d0 0x7f42b41037d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00efc0 tx=0x7f42ac00c3a0 comp rx=0 tx=0).stop 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 shutdown_connections 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42b41022d0 0x7f42b41037d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 --2- 172.21.15.174:0/4224268984 >> v2:172.21.15.174:6800/2183136494 conn(0x7f429c039180 0x7f429c03b640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42b4100ea0 0x7f42b4104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 --2- 172.21.15.174:0/4224268984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42b4101860 0x7f42b4103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 >> 172.21.15.174:0/4224268984 conn(0x7f42b40fb990 msgr2=0x7f42b40fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:46.353 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 shutdown_connections 2024-02-10T06:40:46.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:46.351+0000 7f42ba740700 1 -- 172.21.15.174:0/4224268984 wait complete. 2024-02-10T06:40:46.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:46.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:46.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T06:40:46.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:46 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:46 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:40:46.964 DEBUG:teuthology.orchestra.run.smithi174:osd.0> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.0.service 2024-02-10T06:40:46.966 INFO:tasks.cephadm:Deploying osd.1 on smithi174 with /dev/vg_nvme/lv_3... 2024-02-10T06:40:46.967 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-10T06:40:47.679 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[29878]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[29878]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[29878]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[29878]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[39523]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:47.800 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[39523]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T06:40:47.801 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[39523]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T06:40:47.801 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[39523]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:40:47.801 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:47 smithi184 ceph-mon[33513]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:47 smithi184 ceph-mon[33513]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T06:40:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:47 smithi184 ceph-mon[33513]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T06:40:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:47 smithi184 ceph-mon[33513]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:40:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:47 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.071 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0[43919]: 2024-02-10T06:40:47.881+0000 7f54f61d7700 -1 osd.0 0 waiting for initial osdmap 2024-02-10T06:40:48.071 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:40:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0[43919]: 2024-02-10T06:40:47.885+0000 7f54edfc2700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:40:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[39523]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:40:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[39523]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T06:40:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[39523]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[29878]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[29878]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[29878]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' 2024-02-10T06:40:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:48 smithi184 ceph-mon[33513]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:40:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:48 smithi184 ceph-mon[33513]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T06:40:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:48 smithi184 ceph-mon[33513]: from='osd.0 v2:172.21.15.174:6801/4183537518' entity='osd.0' 2024-02-10T06:40:48.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:49.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:40:49.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:40:49.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[39523]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:49.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[39523]: osd.0 v2:172.21.15.174:6801/4183537518 boot 2024-02-10T06:40:49.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[39523]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T06:40:49.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:49.781 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:40:49.781 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:40:49.781 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[29878]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:49.781 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[29878]: osd.0 v2:172.21.15.174:6801/4183537518 boot 2024-02-10T06:40:49.781 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[29878]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T06:40:49.781 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:49 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:49 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:40:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:49 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:40:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:49 smithi184 ceph-mon[33513]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T06:40:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:49 smithi184 ceph-mon[33513]: osd.0 v2:172.21.15.174:6801/4183537518 boot 2024-02-10T06:40:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:49 smithi184 ceph-mon[33513]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T06:40:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:49 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:40:50.608 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-10T06:40:50.608 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-10T06:40:50.608 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-02-10T06:40:50.608 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-02-10T06:40:50.608 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151293 s, 693 MB/s 2024-02-10T06:40:50.609 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-02-10T06:40:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:51 smithi184 ceph-mon[33513]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:51 smithi184 ceph-mon[33513]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T06:40:51.940 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:51 smithi174 ceph-mon[29878]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:51.940 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:51 smithi174 ceph-mon[29878]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T06:40:51.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:51 smithi174 ceph-mon[39523]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:51.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:51 smithi174 ceph-mon[39523]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T06:40:51.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-02-10T06:40:52.202 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi174 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi174 to 16064M 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[29878]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:52.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi174 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi174 to 16064M 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.595 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:52 smithi174 ceph-mon[39523]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi174 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi174 to 16064M 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:40:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:52 smithi184 ceph-mon[33513]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.837+0000 7fedebab7700 1 -- 172.21.15.174:0/1436034797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4101d70 msgr2=0x7fede4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.837+0000 7fedebab7700 1 --2- 172.21.15.174:0/1436034797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4101d70 0x7fede4102190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede00099c0 tx=0x7fede0009cd0 comp rx=0 tx=0).stop 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.837+0000 7fedebab7700 1 -- 172.21.15.174:0/1436034797 shutdown_connections 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.837+0000 7fedebab7700 1 --2- 172.21.15.174:0/1436034797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede4103910 0x7fede410a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.837+0000 7fedebab7700 1 --2- 172.21.15.174:0/1436034797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4101d70 0x7fede4102190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 --2- 172.21.15.174:0/1436034797 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4102f70 0x7fede41033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 -- 172.21.15.174:0/1436034797 >> 172.21.15.174:0/1436034797 conn(0x7fede40fd330 msgr2=0x7fede40ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 -- 172.21.15.174:0/1436034797 shutdown_connections 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 -- 172.21.15.174:0/1436034797 wait complete. 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 Processor -- start 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 -- start start 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.838+0000 7fedebab7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4101d70 0x7fede4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:53.841 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 0x7fede4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede4103910 0x7fede419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede419ef90 con 0x7fede4102f70 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede419f0d0 con 0x7fede4103910 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede419f260 con 0x7fede4101d70 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9052700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 0x7fede4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4101d70 0x7fede4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9052700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 0x7fede4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55772/0 (socket says 172.21.15.174:55772) 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9052700 1 -- 172.21.15.174:0/1915533140 learned_addr learned my addr 172.21.15.174:0/1915533140 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedea054700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede4103910 0x7fede419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 -- 172.21.15.174:0/1915533140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede4103910 msgr2=0x7fede419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede4103910 0x7fede419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 -- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 msgr2=0x7fede4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 0x7fede4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 -- 172.21.15.174:0/1915533140 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede00095f0 con 0x7fede4101d70 2024-02-10T06:40:53.842 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9052700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 0x7fede4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:40:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fede9853700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4101d70 0x7fede4197400 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fede0009fd0 tx=0x7fede0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede001c070 con 0x7fede4101d70 2024-02-10T06:40:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fede000fd40 con 0x7fede4101d70 2024-02-10T06:40:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede419f4e0 con 0x7fede4101d70 2024-02-10T06:40:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0022c60 con 0x7fede4101d70 2024-02-10T06:40:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.839+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede419f9d0 con 0x7fede4101d70 2024-02-10T06:40:53.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.840+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede4108950 con 0x7fede4101d70 2024-02-10T06:40:53.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.841+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7fede0020070 con 0x7fede4101d70 2024-02-10T06:40:53.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.841+0000 7feddaffd700 1 --2- 172.21.15.174:0/1915533140 >> v2:172.21.15.174:6800/2183136494 conn(0x7fedd006d900 0x7fedd006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:40:53.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.841+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1541+0+0 (secure 0 0 0) 0x7fede008cd80 con 0x7fede4101d70 2024-02-10T06:40:53.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.841+0000 7fede9052700 1 --2- 172.21.15.174:0/1915533140 >> v2:172.21.15.174:6800/2183136494 conn(0x7fedd006d900 0x7fedd006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:40:53.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.842+0000 7fede9052700 1 --2- 172.21.15.174:0/1915533140 >> v2:172.21.15.174:6800/2183136494 conn(0x7fedd006d900 0x7fedd006fdc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fedd4011550 tx=0x7fedd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:40:53.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:53.846+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fede0057b60 con 0x7fede4101d70 2024-02-10T06:40:54.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:40:54.050+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fede410cf40 con 0x7fedd006d900 2024-02-10T06:40:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:54 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:40:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:54 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:40:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:54 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:54 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:40:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:54 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:40:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:54 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:54 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:40:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:54 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:40:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:54 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:40:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:55 smithi184 ceph-mon[33513]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:55 smithi184 ceph-mon[33513]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:55.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:55 smithi174 ceph-mon[29878]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:55.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:55 smithi174 ceph-mon[29878]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:55.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:55 smithi174 ceph-mon[39523]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:40:55.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:55 smithi174 ceph-mon[39523]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[29878]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/689279634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]: dispatch 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]: dispatch 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]': finished 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[29878]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[39523]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/689279634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]: dispatch 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]: dispatch 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]': finished 2024-02-10T06:40:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[39523]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T06:40:57.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:57 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:40:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:57 smithi184 ceph-mon[33513]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/689279634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]: dispatch 2024-02-10T06:40:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]: dispatch 2024-02-10T06:40:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aebb7103-2298-403b-8df6-e078cf073484"}]': finished 2024-02-10T06:40:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:57 smithi184 ceph-mon[33513]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T06:40:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:40:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1654975179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:40:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1654975179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:40:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1654975179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:40:59.721 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:40:59 smithi174 ceph-mon[29878]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:59.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:40:59 smithi174 ceph-mon[39523]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:40:59.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:40:59 smithi184 ceph-mon[33513]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:01 smithi174 ceph-mon[29878]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:01 smithi174 ceph-mon[39523]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:01.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:01 smithi184 ceph-mon[33513]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:03.675 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:03 smithi184 ceph-mon[33513]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:03.675 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T06:41:03.675 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:03.717 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:03 smithi174 ceph-mon[29878]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:03.717 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T06:41:03.717 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:03.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:03 smithi174 ceph-mon[39523]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:03.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T06:41:03.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:04.722 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:04 smithi174 ceph-mon[29878]: Deploying daemon osd.1 on smithi174 2024-02-10T06:41:04.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:04 smithi174 ceph-mon[39523]: Deploying daemon osd.1 on smithi174 2024-02-10T06:41:04.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:04 smithi184 ceph-mon[33513]: Deploying daemon osd.1 on smithi174 2024-02-10T06:41:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:05 smithi174 ceph-mon[29878]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:05 smithi174 ceph-mon[39523]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:05 smithi184 ceph-mon[33513]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:07 smithi174 ceph-mon[29878]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:07 smithi174 ceph-mon[39523]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:07.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:07 smithi184 ceph-mon[33513]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[29878]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:08.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[39523]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:08.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:08.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:08.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:08 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:08 smithi184 ceph-mon[33513]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:08 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:08 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:08 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:11.435 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:11 smithi174 ceph-mon[39523]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:11.435 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:11 smithi174 ceph-mon[29878]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:11 smithi184 ceph-mon[33513]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:12.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.151+0000 7feddaffd700 1 -- 172.21.15.174:0/1915533140 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fede410cf40 con 0x7fedd006d900 2024-02-10T06:41:12.152 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 1 on host 'smithi174' 2024-02-10T06:41:12.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.153+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 >> v2:172.21.15.174:6800/2183136494 conn(0x7fedd006d900 msgr2=0x7fedd006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:12.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.153+0000 7fedebab7700 1 --2- 172.21.15.174:0/1915533140 >> v2:172.21.15.174:6800/2183136494 conn(0x7fedd006d900 0x7fedd006fdc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fedd4011550 tx=0x7fedd4009260 comp rx=0 tx=0).stop 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4101d70 msgr2=0x7fede4197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4101d70 0x7fede4197400 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fede0009fd0 tx=0x7fede0009580 comp rx=0 tx=0).stop 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 shutdown_connections 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 --2- 172.21.15.174:0/1915533140 >> v2:172.21.15.174:6800/2183136494 conn(0x7fedd006d900 0x7fedd006fdc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fede4103910 0x7fede419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fede4101d70 0x7fede4197400 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 --2- 172.21.15.174:0/1915533140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede4102f70 0x7fede4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 >> 172.21.15.174:0/1915533140 conn(0x7fede40fd330 msgr2=0x7fede40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 shutdown_connections 2024-02-10T06:41:12.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:12.154+0000 7fedebab7700 1 -- 172.21.15.174:0/1915533140 wait complete. 2024-02-10T06:41:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[29878]: from='osd.1 v2:172.21.15.174:6805/640275884' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:41:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[29878]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:41:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[39523]: from='osd.1 v2:172.21.15.174:6805/640275884' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:41:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[39523]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:41:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:12 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:12.784 DEBUG:teuthology.orchestra.run.smithi174:osd.1> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.1.service 2024-02-10T06:41:12.786 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_2... 2024-02-10T06:41:12.786 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-10T06:41:12.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:12 smithi184 ceph-mon[33513]: from='osd.1 v2:172.21.15.174:6805/640275884' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:41:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:12 smithi184 ceph-mon[33513]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T06:41:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:12 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:12 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:13.081 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:41:13.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:13.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T06:41:13.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T06:41:13.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: from='osd.1 v2:172.21.15.174:6805/640275884' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: from='osd.1 v2:172.21.15.174:6805/640275884' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:13.479 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:13.480 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:13.480 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: from='osd.1 v2:172.21.15.174:6805/640275884' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:13 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:14.186 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1[50156]: 2024-02-10T06:41:13.832+0000 7f4c73452700 -1 osd.1 0 waiting for initial osdmap 2024-02-10T06:41:14.186 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:13 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1[50156]: 2024-02-10T06:41:13.835+0000 7f4c6b23d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[29878]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[29878]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[29878]: from='osd.1 ' entity='osd.1' 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[39523]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[39523]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T06:41:14.451 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:14.452 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:14.452 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:14 smithi174 ceph-mon[39523]: from='osd.1 ' entity='osd.1' 2024-02-10T06:41:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:14 smithi184 ceph-mon[33513]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:41:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:14 smithi184 ceph-mon[33513]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T06:41:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:14 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:14 smithi184 ceph-mon[33513]: from='osd.1 ' entity='osd.1' 2024-02-10T06:41:15.047 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-10T06:41:15.048 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-10T06:41:15.048 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-02-10T06:41:15.048 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-02-10T06:41:15.048 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136442 s, 769 MB/s 2024-02-10T06:41:15.048 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-02-10T06:41:15.691 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:41:15.691 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:41:15.691 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: osd.1 v2:172.21.15.174:6805/640275884 boot 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: osd.1 v2:172.21.15.174:6805/640275884 boot 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T06:41:15.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: osd.1 v2:172.21.15.174:6805/640275884 boot 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T06:41:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:41:16.007 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-02-10T06:41:16.441 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:41:17.433 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:17 smithi174 ceph-mon[29878]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:17.434 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:17 smithi174 ceph-mon[29878]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T06:41:17.434 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:17 smithi174 ceph-mon[39523]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:17.434 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:17 smithi174 ceph-mon[39523]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T06:41:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:17 smithi184 ceph-mon[33513]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:17 smithi184 ceph-mon[33513]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 -- 172.21.15.174:0/2321971421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc1018b0 msgr2=0x7f01fc109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 --2- 172.21.15.174:0/2321971421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc1018b0 0x7f01fc109150 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009f90 tx=0x7f01f400b330 comp rx=0 tx=0).stop 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 -- 172.21.15.174:0/2321971421 shutdown_connections 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 --2- 172.21.15.174:0/2321971421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc1018b0 0x7f01fc109150 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 --2- 172.21.15.174:0/2321971421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc102320 0x7f01fc102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 --2- 172.21.15.174:0/2321971421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01fc100ef0 0x7f01fc101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 -- 172.21.15.174:0/2321971421 >> 172.21.15.174:0/2321971421 conn(0x7f01fc0fb9c0 msgr2=0x7f01fc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 -- 172.21.15.174:0/2321971421 shutdown_connections 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.182+0000 7f02029bd700 1 -- 172.21.15.174:0/2321971421 wait complete. 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 Processor -- start 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 -- start start 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01fc100ef0 0x7f01fc197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc1018b0 0x7f01fc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:19.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 0x7f01fc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19f110 con 0x7f01fc1018b0 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19f250 con 0x7f01fc100ef0 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f02029bd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19f3e0 con 0x7f01fc102320 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc1018b0 0x7f01fc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f0200f5a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 0x7f01fc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f0200f5a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 0x7f01fc19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:42550/0 (socket says 172.21.15.174:42550) 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.183+0000 7f0200f5a700 1 -- 172.21.15.174:0/340874823 learned_addr learned my addr 172.21.15.174:0/340874823 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fbfff700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01fc100ef0 0x7f01fc197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fb7fe700 1 -- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 msgr2=0x7f01fc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fb7fe700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 0x7f01fc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fb7fe700 1 -- 172.21.15.174:0/340874823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01fc100ef0 msgr2=0x7f01fc197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fb7fe700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01fc100ef0 0x7f01fc197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fb7fe700 1 -- 172.21.15.174:0/340874823 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0095f0 con 0x7f01fc1018b0 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f0200f5a700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 0x7f01fc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:41:19.186 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01fb7fe700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc1018b0 0x7f01fc197ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f01f000b480 tx=0x7f01f000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:41:19.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0011830 con 0x7f01fc1018b0 2024-02-10T06:41:19.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4009c40 con 0x7f01fc1018b0 2024-02-10T06:41:19.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f01f0011e40 con 0x7f01fc1018b0 2024-02-10T06:41:19.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0010dc0 con 0x7f01fc1018b0 2024-02-10T06:41:19.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.184+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc19fa30 con 0x7f01fc1018b0 2024-02-10T06:41:19.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.185+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc108ae0 con 0x7f01fc1018b0 2024-02-10T06:41:19.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.186+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 14) v1 ==== 91316+0+0 (secure 0 0 0) 0x7f01f0011990 con 0x7f01fc1018b0 2024-02-10T06:41:19.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.186+0000 7f01f97fa700 1 --2- 172.21.15.174:0/340874823 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01e406d900 0x7f01e406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:19.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.186+0000 7f01fbfff700 1 --2- 172.21.15.174:0/340874823 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01e406d900 0x7f01e406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:19.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.186+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 1833+0+0 (secure 0 0 0) 0x7f01f008b9f0 con 0x7f01fc1018b0 2024-02-10T06:41:19.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.186+0000 7f01fbfff700 1 --2- 172.21.15.174:0/340874823 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01e406d900 0x7f01e406fdc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005fd0 tx=0x7f01ec009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:41:19.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.191+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01f0056510 con 0x7f01fc1018b0 2024-02-10T06:41:19.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:19.396+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f01fc10bf90 con 0x7f01e406d900 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi174 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi174 to 8032M 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi174 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi174 to 8032M 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi174 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi174 to 8032M 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:20 smithi184 ceph-mon[33513]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:41:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:41:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:41:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[29878]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:41:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:41:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:41:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:20.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[39523]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:41:20.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:41:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:41:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:21.359 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:21 smithi174 ceph-mon[29878]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:21.359 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:21 smithi174 ceph-mon[39523]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:21.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:21 smithi184 ceph-mon[33513]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:22.577 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3981979824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11f1e7b8-5292-4c92-8823-1e9387e69d7f"}]: dispatch 2024-02-10T06:41:22.577 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3981979824' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11f1e7b8-5292-4c92-8823-1e9387e69d7f"}]': finished 2024-02-10T06:41:22.577 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[29878]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T06:41:22.577 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:22.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3981979824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11f1e7b8-5292-4c92-8823-1e9387e69d7f"}]: dispatch 2024-02-10T06:41:22.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3981979824' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11f1e7b8-5292-4c92-8823-1e9387e69d7f"}]': finished 2024-02-10T06:41:22.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[39523]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T06:41:22.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:22.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3981979824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11f1e7b8-5292-4c92-8823-1e9387e69d7f"}]: dispatch 2024-02-10T06:41:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3981979824' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11f1e7b8-5292-4c92-8823-1e9387e69d7f"}]': finished 2024-02-10T06:41:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:22 smithi184 ceph-mon[33513]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T06:41:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:23 smithi184 ceph-mon[33513]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/648035733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:41:23.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:23 smithi174 ceph-mon[29878]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:23.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/648035733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:41:23.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:23 smithi174 ceph-mon[39523]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:23.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/648035733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:41:25.716 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:25 smithi174 ceph-mon[29878]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:25.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:25 smithi174 ceph-mon[39523]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:25.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:25 smithi184 ceph-mon[33513]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:27.655 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:27 smithi174 ceph-mon[29878]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:27.655 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:27 smithi174 ceph-mon[39523]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:27 smithi184 ceph-mon[33513]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:28 smithi184 ceph-mon[33513]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T06:41:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:28 smithi174 ceph-mon[29878]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T06:41:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:28 smithi174 ceph-mon[39523]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T06:41:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:29.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:29 smithi174 ceph-mon[29878]: Deploying daemon osd.2 on smithi174 2024-02-10T06:41:29.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:29 smithi174 ceph-mon[39523]: Deploying daemon osd.2 on smithi174 2024-02-10T06:41:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:29 smithi184 ceph-mon[33513]: Deploying daemon osd.2 on smithi174 2024-02-10T06:41:30.920 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:30 smithi174 ceph-mon[29878]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:30.921 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:30 smithi174 ceph-mon[39523]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:30 smithi184 ceph-mon[33513]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:33.732 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:33 smithi174 ceph-mon[39523]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:33.732 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:33 smithi174 ceph-mon[29878]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:33.826 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:33 smithi184 ceph-mon[33513]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[29878]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[39523]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:35.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:35 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:35 smithi184 ceph-mon[33513]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:35 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:35 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:35 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:37 smithi174 ceph-mon[29878]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:37 smithi174 ceph-mon[39523]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:37.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:37 smithi184 ceph-mon[33513]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:37.896 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.895+0000 7f01f97fa700 1 -- 172.21.15.174:0/340874823 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f01fc10bf90 con 0x7f01e406d900 2024-02-10T06:41:37.897 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01e406d900 msgr2=0x7f01e406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 --2- 172.21.15.174:0/340874823 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01e406d900 0x7f01e406fdc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005fd0 tx=0x7f01ec009310 comp rx=0 tx=0).stop 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc1018b0 msgr2=0x7f01fc197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc1018b0 0x7f01fc197ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f01f000b480 tx=0x7f01f000b790 comp rx=0 tx=0).stop 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 shutdown_connections 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01fc102320 0x7f01fc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 --2- 172.21.15.174:0/340874823 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01e406d900 0x7f01e406fdc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01fc100ef0 0x7f01fc197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 --2- 172.21.15.174:0/340874823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01fc1018b0 0x7f01fc197ac0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 >> 172.21.15.174:0/340874823 conn(0x7f01fc0fb9c0 msgr2=0x7f01fc0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 shutdown_connections 2024-02-10T06:41:37.899 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:37.898+0000 7f02029bd700 1 -- 172.21.15.174:0/340874823 wait complete. 2024-02-10T06:41:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:38 smithi174 ceph-mon[39523]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:41:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:38 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:38 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:38.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:38 smithi174 ceph-mon[29878]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:41:38.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:38.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:38.522 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.2.service 2024-02-10T06:41:38.524 INFO:tasks.cephadm:Deploying osd.3 on smithi174 with /dev/vg_nvme/lv_1... 2024-02-10T06:41:38.524 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-10T06:41:38.814 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:41:38.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:38 smithi184 ceph-mon[33513]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T06:41:38.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:38 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:38 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T06:41:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T06:41:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:39.491 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2[56431]: 2024-02-10T06:41:39.390+0000 7f81b26fc700 -1 osd.2 0 waiting for initial osdmap 2024-02-10T06:41:39.491 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2[56431]: 2024-02-10T06:41:39.393+0000 7f81ae4ef700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:39.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:39 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[29878]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[29878]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[39523]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[39523]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:40.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:40 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:40.863 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-10T06:41:40.864 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-10T06:41:40.864 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-02-10T06:41:40.864 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-02-10T06:41:40.864 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151757 s, 691 MB/s 2024-02-10T06:41:40.864 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-02-10T06:41:40.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:40 smithi184 ceph-mon[33513]: from='osd.2 v2:172.21.15.174:6809/2318790055' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:41:40.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:40 smithi184 ceph-mon[33513]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T06:41:40.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:40 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:40.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:40 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: osd.2 v2:172.21.15.174:6809/2318790055 boot 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: osd.2 v2:172.21.15.174:6809/2318790055 boot 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:41 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: osd.2 v2:172.21.15.174:6809/2318790055 boot 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:41:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:41 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:41:42.017 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-02-10T06:41:42.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:42 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T06:41:42.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:42 smithi174 ceph-mon[39523]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T06:41:42.652 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:42 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:41:42.652 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:42 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T06:41:42.652 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:42 smithi174 ceph-mon[29878]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T06:41:42.653 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:42 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:41:42.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:42 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T06:41:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:42 smithi184 ceph-mon[33513]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T06:41:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:42 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:41:42.924 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:41:43.182 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:42 smithi174 sudo[58174]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:41:43.182 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:42 smithi174 sudo[58174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:43.182 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:42 smithi174 sudo[58174]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:43.182 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:42 smithi174 sudo[58174]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:43.480 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[58227]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:41:43.480 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[58227]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:43.480 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[58227]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:43.480 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[58227]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[29878]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[29878]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[29878]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[39523]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[39523]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[39523]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T06:41:43.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:43 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:43.739 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[59897]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:41:43.739 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[59897]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:43.739 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[59897]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:43.739 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[59897]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:43 smithi184 ceph-mon[33513]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:43 smithi184 ceph-mon[33513]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:41:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:43 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:41:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:43 smithi184 ceph-mon[33513]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T06:41:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:43 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60690]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60690]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60690]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60690]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60677]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60677]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:44.009 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:41:43 smithi174 sudo[60677]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:44.523 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60734]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:41:44.524 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60734]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:44.524 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60734]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:44.524 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60731]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:41:44.524 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:44.524 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60731]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:44.524 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60731]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:44.775 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60734]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:44.776 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60757]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:41:44.776 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60757]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:44.776 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60757]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:44.776 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 sudo[60757]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36305]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:41:45.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:45.036 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36305]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:45.036 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36305]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.098 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:45.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:45.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.111+0000 7fc387ee7700 1 -- 172.21.15.174:0/1388503091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3801038e0 msgr2=0x7fc38010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:45.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.111+0000 7fc387ee7700 1 --2- 172.21.15.174:0/1388503091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3801038e0 0x7fc38010a190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b0d0 tx=0x7fc37800b3e0 comp rx=0 tx=0).stop 2024-02-10T06:41:45.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 -- 172.21.15.174:0/1388503091 shutdown_connections 2024-02-10T06:41:45.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 --2- 172.21.15.174:0/1388503091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3801038e0 0x7fc38010a190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:45.112 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 --2- 172.21.15.174:0/1388503091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc380101d40 0x7fc380102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:45.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 --2- 172.21.15.174:0/1388503091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc380102f40 0x7fc3801033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:45.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 -- 172.21.15.174:0/1388503091 >> 172.21.15.174:0/1388503091 conn(0x7fc3800fd300 msgr2=0x7fc3800ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:41:45.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 -- 172.21.15.174:0/1388503091 shutdown_connections 2024-02-10T06:41:45.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 -- 172.21.15.174:0/1388503091 wait complete. 2024-02-10T06:41:45.114 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 Processor -- start 2024-02-10T06:41:45.114 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 -- start start 2024-02-10T06:41:45.114 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc380101d40 0x7fc380075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 0x7fc380074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.112+0000 7fc387ee7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3801038e0 0x7fc380074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc387ee7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380074dd0 con 0x7fc380102f40 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc387ee7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380074f40 con 0x7fc380101d40 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc387ee7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3801a1520 con 0x7fc3801038e0 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc385482700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 0x7fc380074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3801038e0 0x7fc380074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc385482700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 0x7fc380074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43018/0 (socket says 172.21.15.174:43018) 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc385482700 1 -- 172.21.15.174:0/235739404 learned_addr learned my addr 172.21.15.174:0/235739404 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc385c83700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc380101d40 0x7fc380075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 -- 172.21.15.174:0/235739404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc380101d40 msgr2=0x7fc380075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc380101d40 0x7fc380075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 -- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 msgr2=0x7fc380074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:41:45.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 0x7fc380074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 -- 172.21.15.174:0/235739404 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc378009d00 con 0x7fc3801038e0 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc385482700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 0x7fc380074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc386484700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3801038e0 0x7fc380074650 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b770 tx=0x7fc378008f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800e040 con 0x7fc3801038e0 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3801a17a0 con 0x7fc3801038e0 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc37801d920 con 0x7fc3801038e0 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801b9f0 con 0x7fc3801038e0 2024-02-10T06:41:45.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.113+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3801a1c60 con 0x7fc3801038e0 2024-02-10T06:41:45.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.114+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc380108920 con 0x7fc3801038e0 2024-02-10T06:41:45.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.115+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7fc37801d470 con 0x7fc3801038e0 2024-02-10T06:41:45.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.115+0000 7fc376ffd700 1 --2- 172.21.15.174:0/235739404 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc36c06d990 0x7fc36c06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:41:45.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.115+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2511+0+0 (secure 0 0 0) 0x7fc37808d870 con 0x7fc3801038e0 2024-02-10T06:41:45.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.115+0000 7fc385c83700 1 --2- 172.21.15.174:0/235739404 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc36c06d990 0x7fc36c06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:41:45.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.116+0000 7fc385c83700 1 --2- 172.21.15.174:0/235739404 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc36c06d990 0x7fc36c06fe50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc380102da0 tx=0x7fc37c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:41:45.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.120+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc378058010 con 0x7fc3801038e0 2024-02-10T06:41:45.338 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:41:45.337+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc38010cf90 con 0x7fc36c06d990 2024-02-10T06:41:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36308]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:41:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36308]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 sudo[36308]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:45.821 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60905]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T06:41:45.821 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60905]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:45.821 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60905]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:45.821 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60905]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60947]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60947]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60947]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 sudo[60947]: pam_unix(sudo:session): session closed for user root 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi174 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi174 to 5354M 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.080 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:41:46.081 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi174 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi174 to 5354M 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:46.082 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:41:46.083 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi174 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi174 to 5354M 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:41:46.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:46.921 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[29878]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.921 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[29878]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:41:46.921 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[29878]: Cluster is now healthy 2024-02-10T06:41:46.921 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[29878]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.921 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[29878]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:46.921 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[39523]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[39523]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:41:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[39523]: Cluster is now healthy 2024-02-10T06:41:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[39523]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:46 smithi174 ceph-mon[39523]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:46 smithi184 ceph-mon[33513]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T06:41:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:46 smithi184 ceph-mon[33513]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:41:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:46 smithi184 ceph-mon[33513]: Cluster is now healthy 2024-02-10T06:41:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:46 smithi184 ceph-mon[33513]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T06:41:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:46 smithi184 ceph-mon[33513]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3026706561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6db6715d-ad5c-42a4-b051-0d2fdd2fef2a"}]: dispatch 2024-02-10T06:41:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3026706561' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6db6715d-ad5c-42a4-b051-0d2fdd2fef2a"}]': finished 2024-02-10T06:41:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:48 smithi184 ceph-mon[33513]: osdmap e22: 4 total, 3 up, 4 in 2024-02-10T06:41:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:41:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:48 smithi184 ceph-mon[33513]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2896716982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:41:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3026706561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6db6715d-ad5c-42a4-b051-0d2fdd2fef2a"}]: dispatch 2024-02-10T06:41:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3026706561' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6db6715d-ad5c-42a4-b051-0d2fdd2fef2a"}]': finished 2024-02-10T06:41:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[39523]: osdmap e22: 4 total, 3 up, 4 in 2024-02-10T06:41:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:41:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[39523]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2896716982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3026706561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6db6715d-ad5c-42a4-b051-0d2fdd2fef2a"}]: dispatch 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3026706561' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6db6715d-ad5c-42a4-b051-0d2fdd2fef2a"}]': finished 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[29878]: osdmap e22: 4 total, 3 up, 4 in 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[29878]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2896716982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:41:51.596 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:51 smithi174 ceph-mon[29878]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:51.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:51 smithi174 ceph-mon[39523]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:51 smithi184 ceph-mon[33513]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:41:53.548 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:53 smithi174 ceph-mon[29878]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-02-10T06:41:53.548 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:53 smithi174 ceph-mon[39523]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-02-10T06:41:53.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:53 smithi184 ceph-mon[33513]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-02-10T06:41:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:55 smithi174 ceph-mon[39523]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-02-10T06:41:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:55 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T06:41:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:55 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:55 smithi174 ceph-mon[29878]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-02-10T06:41:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:55 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T06:41:55.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:55 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:55.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:55 smithi184 ceph-mon[33513]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-02-10T06:41:55.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:55 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T06:41:55.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:55 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:41:56.684 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:56 smithi174 ceph-mon[29878]: Deploying daemon osd.3 on smithi174 2024-02-10T06:41:56.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:56 smithi174 ceph-mon[39523]: Deploying daemon osd.3 on smithi174 2024-02-10T06:41:56.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:56 smithi184 ceph-mon[33513]: Deploying daemon osd.3 on smithi174 2024-02-10T06:41:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:57 smithi174 ceph-mon[29878]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-10T06:41:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:57 smithi174 ceph-mon[39523]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-10T06:41:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:57 smithi184 ceph-mon[33513]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-10T06:41:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:41:58 smithi184 ceph-mon[33513]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 892 B/s rd, 25 KiB/s wr, 2 op/s 2024-02-10T06:41:59.230 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:41:58 smithi174 ceph-mon[29878]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 892 B/s rd, 25 KiB/s wr, 2 op/s 2024-02-10T06:41:59.230 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:41:58 smithi174 ceph-mon[39523]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 892 B/s rd, 25 KiB/s wr, 2 op/s 2024-02-10T06:42:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:01 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:01 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:01 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:01 smithi184 ceph-mon[33513]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[29878]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:01 smithi174 ceph-mon[39523]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='osd.3 v2:172.21.15.174:6813/2745280023' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='osd.3 v2:172.21.15.174:6813/2745280023' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='osd.3 v2:172.21.15.174:6813/2745280023' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:03.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:04.092 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.090+0000 7fc376ffd700 1 -- 172.21.15.174:0/235739404 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc38010cf90 con 0x7fc36c06d990 2024-02-10T06:42:04.092 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 3 on host 'smithi174' 2024-02-10T06:42:04.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc36c06d990 msgr2=0x7fc36c06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:04.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 --2- 172.21.15.174:0/235739404 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc36c06d990 0x7fc36c06fe50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc380102da0 tx=0x7fc37c00a380 comp rx=0 tx=0).stop 2024-02-10T06:42:04.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3801038e0 msgr2=0x7fc380074650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:04.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3801038e0 0x7fc380074650 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b770 tx=0x7fc378008f40 comp rx=0 tx=0).stop 2024-02-10T06:42:04.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 shutdown_connections 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3801038e0 0x7fc380074650 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 --2- 172.21.15.174:0/235739404 >> v2:172.21.15.174:6800/2183136494 conn(0x7fc36c06d990 0x7fc36c06fe50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc380101d40 0x7fc380075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 --2- 172.21.15.174:0/235739404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc380102f40 0x7fc380074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 >> 172.21.15.174:0/235739404 conn(0x7fc3800fd300 msgr2=0x7fc3800fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.092+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 shutdown_connections 2024-02-10T06:42:04.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:42:04.093+0000 7fc387ee7700 1 -- 172.21.15.174:0/235739404 wait complete. 2024-02-10T06:42:04.771 DEBUG:teuthology.orchestra.run.smithi174:osd.3> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.3.service 2024-02-10T06:42:04.773 INFO:tasks.cephadm:Deploying osd.4 on smithi184 with /dev/vg_nvme/lv_4... 2024-02-10T06:42:04.774 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-10T06:42:05.020 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='osd.3 v2:172.21.15.174:6813/2745280023' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: osdmap e23: 4 total, 3 up, 4 in 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:05.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[39523]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='osd.3 v2:172.21.15.174:6813/2745280023' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: osdmap e23: 4 total, 3 up, 4 in 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:05.079 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-mon[29878]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:42:05.080 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T06:42:04.842+0000 7fac783fa700 -1 osd.3 0 waiting for initial osdmap 2024-02-10T06:42:05.080 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 06:42:04 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T06:42:04.845+0000 7fac741ed700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:42:05.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T06:42:05.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='osd.3 v2:172.21.15.174:6813/2745280023' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: osdmap e23: 4 total, 3 up, 4 in 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:04 smithi184 ceph-mon[33513]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:42:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:05 smithi184 ceph-mon[33513]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:42:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:05 smithi184 ceph-mon[33513]: osdmap e24: 4 total, 3 up, 4 in 2024-02-10T06:42:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:05 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:05 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:05 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[29878]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[29878]: osdmap e24: 4 total, 3 up, 4 in 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[39523]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[39523]: osdmap e24: 4 total, 3 up, 4 in 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.201 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:06.202 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:05 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:07.067 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-10T06:42:07.067 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-10T06:42:07.068 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-10T06:42:07.068 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-10T06:42:07.068 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114103 s, 919 MB/s 2024-02-10T06:42:07.068 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-10T06:42:07.108 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:42:07.108 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:42:07.108 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[29878]: osd.3 v2:172.21.15.174:6813/2745280023 boot 2024-02-10T06:42:07.108 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[29878]: osdmap e25: 4 total, 4 up, 4 in 2024-02-10T06:42:07.108 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:07.108 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[29878]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:42:07.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:42:07.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:42:07.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[39523]: osd.3 v2:172.21.15.174:6813/2745280023 boot 2024-02-10T06:42:07.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[39523]: osdmap e25: 4 total, 4 up, 4 in 2024-02-10T06:42:07.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:07.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:06 smithi174 ceph-mon[39523]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:42:07.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:06 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:42:07.121 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:06 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:42:07.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:06 smithi184 ceph-mon[33513]: osd.3 v2:172.21.15.174:6813/2745280023 boot 2024-02-10T06:42:07.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:06 smithi184 ceph-mon[33513]: osdmap e25: 4 total, 4 up, 4 in 2024-02-10T06:42:07.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:06 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:42:07.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:06 smithi184 ceph-mon[33513]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-10T06:42:07.673 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-02-10T06:42:07.911 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: osdmap e26: 4 total, 4 up, 4 in 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:08.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:08.035 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:07 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: osdmap e26: 4 total, 4 up, 4 in 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: osdmap e26: 4 total, 4 up, 4 in 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:08.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:07 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:09.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:08 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi174 2024-02-10T06:42:09.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:08 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi174 to 4016M 2024-02-10T06:42:09.068 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:08 smithi184 ceph-mon[33513]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:08 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi174 2024-02-10T06:42:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:08 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi174 to 4016M 2024-02-10T06:42:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:08 smithi174 ceph-mon[39523]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:08 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi174 2024-02-10T06:42:09.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:08 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi174 to 4016M 2024-02-10T06:42:09.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:08 smithi174 ceph-mon[29878]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:09.612 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 -- 172.21.15.184:0/1709097874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c1027f0 msgr2=0x7fec8c1090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 --2- 172.21.15.184:0/1709097874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c1027f0 0x7fec8c1090a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b0d0 tx=0x7fec8400b3e0 comp rx=0 tx=0).stop 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 -- 172.21.15.184:0/1709097874 shutdown_connections 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 --2- 172.21.15.184:0/1709097874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c1027f0 0x7fec8c1090a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 --2- 172.21.15.184:0/1709097874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c100c50 0x7fec8c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 --2- 172.21.15.184:0/1709097874 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec8c101e50 0x7fec8c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 -- 172.21.15.184:0/1709097874 >> 172.21.15.184:0/1709097874 conn(0x7fec8c0fc210 msgr2=0x7fec8c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:42:09.613 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 -- 172.21.15.184:0/1709097874 shutdown_connections 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.602+0000 7fec93bce700 1 -- 172.21.15.184:0/1709097874 wait complete. 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 Processor -- start 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 -- start start 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec8c100c50 0x7fec8c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 0x7fec8c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c1027f0 0x7fec8c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c19f030 con 0x7fec8c1027f0 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c19f1a0 con 0x7fec8c101e50 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec93bce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c19f360 con 0x7fec8c100c50 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 0x7fec8c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 0x7fec8c19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:42286/0 (socket says 172.21.15.184:42286) 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 -- 172.21.15.184:0/2604864099 learned_addr learned my addr 172.21.15.184:0/2604864099 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec9216b700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c1027f0 0x7fec8c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:09.614 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec9196a700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec8c100c50 0x7fec8c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 -- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec8c100c50 msgr2=0x7fec8c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec8c100c50 0x7fec8c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 -- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c1027f0 msgr2=0x7fec8c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c1027f0 0x7fec8c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.603+0000 7fec91169700 1 -- 172.21.15.184:0/2604864099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84009d00 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.604+0000 7fec91169700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 0x7fec8c19c250 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00b530 tx=0x7fec7c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.604+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c011830 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.604+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c19f640 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.604+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fec7c011e40 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.604+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c010e80 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.604+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c19fbc0 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.605+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c107830 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.605+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7fec7c011990 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.605+0000 7fec82ffd700 1 --2- 172.21.15.184:0/2604864099 >> v2:172.21.15.174:6800/2183136494 conn(0x7fec7806d990 0x7fec7806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.605+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 2803+0+0 (secure 0 0 0) 0x7fec7c08bf60 con 0x7fec8c101e50 2024-02-10T06:42:09.615 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.606+0000 7fec9196a700 1 --2- 172.21.15.184:0/2604864099 >> v2:172.21.15.174:6800/2183136494 conn(0x7fec7806d990 0x7fec7806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:09.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.606+0000 7fec9196a700 1 --2- 172.21.15.184:0/2604864099 >> v2:172.21.15.174:6800/2183136494 conn(0x7fec7806d990 0x7fec7806fe50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec8c101cb0 tx=0x7fec8800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:42:09.616 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.610+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec7c056690 con 0x7fec8c101e50 2024-02-10T06:42:09.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:09.816+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fec8c10bf90 con 0x7fec7806d990 2024-02-10T06:42:10.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:09 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:42:10.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:09 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:42:10.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:09 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:09 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:42:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:09 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:42:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:09 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:10.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:09 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:42:10.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:09 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:42:10.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:09 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:11.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:10 smithi184 ceph-mon[33513]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:42:11.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:10 smithi184 ceph-mon[33513]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:10 smithi174 ceph-mon[29878]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:42:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:10 smithi174 ceph-mon[29878]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:10 smithi174 ceph-mon[39523]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:42:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:10 smithi174 ceph-mon[39523]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/34511976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]: dispatch 2024-02-10T06:42:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]: dispatch 2024-02-10T06:42:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]': finished 2024-02-10T06:42:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: osdmap e27: 5 total, 4 up, 5 in 2024-02-10T06:42:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/2669487818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/34511976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]: dispatch 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]: dispatch 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]': finished 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: osdmap e27: 5 total, 4 up, 5 in 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/2669487818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:42:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/34511976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]: dispatch 2024-02-10T06:42:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]: dispatch 2024-02-10T06:42:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3df6af28-331e-466d-ba68-0c7db4e6a616"}]': finished 2024-02-10T06:42:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: osdmap e27: 5 total, 4 up, 5 in 2024-02-10T06:42:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/2669487818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:42:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:15 smithi174 ceph-mon[29878]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:15 smithi174 ceph-mon[39523]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:15.785 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:15 smithi184 ceph-mon[33513]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:17.725 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:17 smithi184 ceph-mon[33513]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:17.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:17 smithi174 ceph-mon[29878]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:17 smithi174 ceph-mon[39523]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:19.041 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:18 smithi184 ceph-mon[33513]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:18 smithi174 ceph-mon[29878]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:18 smithi174 ceph-mon[39523]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:19.951 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T06:42:19.951 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:19 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:19.952 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:19 smithi184 ceph-mon[33513]: Deploying daemon osd.4 on smithi184 2024-02-10T06:42:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T06:42:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:19 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:19 smithi174 ceph-mon[29878]: Deploying daemon osd.4 on smithi184 2024-02-10T06:42:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T06:42:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:19 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:19 smithi174 ceph-mon[39523]: Deploying daemon osd.4 on smithi184 2024-02-10T06:42:21.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:20 smithi184 ceph-mon[33513]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:20 smithi174 ceph-mon[29878]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:20 smithi174 ceph-mon[39523]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:23.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:23 smithi184 ceph-mon[33513]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:23.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:23 smithi174 ceph-mon[29878]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:23.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:23 smithi174 ceph-mon[39523]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:25 smithi184 ceph-mon[33513]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[29878]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:25.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[39523]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:25.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:27.716 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:27 smithi184 ceph-mon[33513]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:27.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:27 smithi174 ceph-mon[29878]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:27.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:27 smithi174 ceph-mon[39523]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:27.829 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.828+0000 7fec82ffd700 1 -- 172.21.15.184:0/2604864099 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec8c10bf90 con 0x7fec7806d990 2024-02-10T06:42:27.829 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 4 on host 'smithi184' 2024-02-10T06:42:27.833 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 >> v2:172.21.15.174:6800/2183136494 conn(0x7fec7806d990 msgr2=0x7fec7806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:27.833 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 --2- 172.21.15.184:0/2604864099 >> v2:172.21.15.174:6800/2183136494 conn(0x7fec7806d990 0x7fec7806fe50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec8c101cb0 tx=0x7fec8800a3b0 comp rx=0 tx=0).stop 2024-02-10T06:42:27.833 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 msgr2=0x7fec8c19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:27.833 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 0x7fec8c19c250 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00b530 tx=0x7fec7c00b840 comp rx=0 tx=0).stop 2024-02-10T06:42:27.833 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 shutdown_connections 2024-02-10T06:42:27.833 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c1027f0 0x7fec8c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:27.834 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec8c100c50 0x7fec8c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:27.834 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 --2- 172.21.15.184:0/2604864099 >> v2:172.21.15.174:6800/2183136494 conn(0x7fec7806d990 0x7fec7806fe50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:27.834 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 --2- 172.21.15.184:0/2604864099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec8c101e50 0x7fec8c19c250 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:27.834 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 >> 172.21.15.184:0/2604864099 conn(0x7fec8c0fc210 msgr2=0x7fec8c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:42:27.834 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 shutdown_connections 2024-02-10T06:42:27.834 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:27.832+0000 7fec93bce700 1 -- 172.21.15.184:0/2604864099 wait complete. 2024-02-10T06:42:28.504 DEBUG:teuthology.orchestra.run.smithi184:osd.4> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.4.service 2024-02-10T06:42:28.506 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-02-10T06:42:28.506 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[29878]: from='osd.4 v2:172.21.15.184:6800/3815653813' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[29878]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[39523]: from='osd.4 v2:172.21.15.184:6800/3815653813' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T06:42:28.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[39523]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T06:42:28.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.788 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:42:28.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:28 smithi184 ceph-mon[33513]: from='osd.4 v2:172.21.15.184:6800/3815653813' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T06:42:28.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:28 smithi184 ceph-mon[33513]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T06:42:28.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:28.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:29.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:29.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T06:42:29.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: from='osd.4 v2:172.21.15.184:6800/3815653813' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:29.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: osdmap e28: 5 total, 4 up, 5 in 2024-02-10T06:42:29.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:29.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:29.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:42:29.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: osdmap e29: 5 total, 4 up, 5 in 2024-02-10T06:42:29.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:29.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:29.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T06:42:29.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: from='osd.4 v2:172.21.15.184:6800/3815653813' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:29.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: osdmap e28: 5 total, 4 up, 5 in 2024-02-10T06:42:29.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:29.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:29.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:42:29.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: osdmap e29: 5 total, 4 up, 5 in 2024-02-10T06:42:29.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:29 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:29.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:29.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T06:42:29.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: from='osd.4 v2:172.21.15.184:6800/3815653813' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:29.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: osdmap e28: 5 total, 4 up, 5 in 2024-02-10T06:42:29.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:29.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:29.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:42:29.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: osdmap e29: 5 total, 4 up, 5 in 2024-02-10T06:42:29.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:29.875 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T06:42:29.423+0000 7f8796eb1700 -1 osd.4 0 waiting for initial osdmap 2024-02-10T06:42:29.875 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 06:42:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T06:42:29.427+0000 7f8792ca4700 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:42:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-10T06:42:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-10T06:42:31.509 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-10T06:42:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-10T06:42:31.509 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122363 s, 857 MB/s 2024-02-10T06:42:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-10T06:42:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:31 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:42:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:31 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:42:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:31 smithi184 ceph-mon[33513]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:31 smithi184 ceph-mon[33513]: osd.4 v2:172.21.15.184:6800/3815653813 boot 2024-02-10T06:42:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:31 smithi184 ceph-mon[33513]: osdmap e30: 5 total, 5 up, 5 in 2024-02-10T06:42:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:31 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[29878]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[29878]: osd.4 v2:172.21.15.184:6800/3815653813 boot 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[29878]: osdmap e30: 5 total, 5 up, 5 in 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:42:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[39523]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-10T06:42:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[39523]: osd.4 v2:172.21.15.184:6800/3815653813 boot 2024-02-10T06:42:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[39523]: osdmap e30: 5 total, 5 up, 5 in 2024-02-10T06:42:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:31 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:42:32.275 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-02-10T06:42:32.533 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:32 smithi184 ceph-mon[33513]: osdmap e31: 5 total, 5 up, 5 in 2024-02-10T06:42:32.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:32 smithi174 ceph-mon[29878]: osdmap e31: 5 total, 5 up, 5 in 2024-02-10T06:42:32.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:32 smithi174 ceph-mon[39523]: osdmap e31: 5 total, 5 up, 5 in 2024-02-10T06:42:32.828 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:42:33.584 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:33 smithi184 ceph-mon[33513]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:33.585 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:33 smithi184 ceph-mon[33513]: osdmap e32: 5 total, 5 up, 5 in 2024-02-10T06:42:33.585 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:33 smithi184 ceph-mon[33513]: osdmap e33: 5 total, 5 up, 5 in 2024-02-10T06:42:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:33 smithi174 ceph-mon[29878]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:33 smithi174 ceph-mon[29878]: osdmap e32: 5 total, 5 up, 5 in 2024-02-10T06:42:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:33 smithi174 ceph-mon[29878]: osdmap e33: 5 total, 5 up, 5 in 2024-02-10T06:42:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:33 smithi174 ceph-mon[39523]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:33 smithi174 ceph-mon[39523]: osdmap e32: 5 total, 5 up, 5 in 2024-02-10T06:42:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:33 smithi174 ceph-mon[39523]: osdmap e33: 5 total, 5 up, 5 in 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: osdmap e34: 5 total, 5 up, 5 in 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi184 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi184 to 17088M 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:35.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:35.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:35.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:35.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:35.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:35.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:34 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: osdmap e34: 5 total, 5 up, 5 in 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi184 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi184 to 17088M 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: osdmap e34: 5 total, 5 up, 5 in 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi184 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi184 to 17088M 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:42:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:34 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2762632823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4101e50 msgr2=0x7f76f41022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2762632823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4101e50 0x7f76f41022b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76e80099c0 tx=0x7f76e8009cd0 comp rx=0 tx=0).stop 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2762632823 shutdown_connections 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2762632823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f41027f0 0x7f76f41090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2762632823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f4100c50 0x7f76f4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2762632823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4101e50 0x7f76f41022b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.376+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2762632823 >> 172.21.15.184:0/2762632823 conn(0x7f76f40fc1f0 msgr2=0x7f76f40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.377+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2762632823 shutdown_connections 2024-02-10T06:42:35.378 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.377+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2762632823 wait complete. 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.377+0000 7f76f8f7f700 1 Processor -- start 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.377+0000 7f76f8f7f700 1 -- start start 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f8f7f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 0x7f76f4196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f8f7f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f4101e50 0x7f76f4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f8f7f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f41027f0 0x7f76f419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f8f7f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419dea0 con 0x7f76f4101e50 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f8f7f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419dfe0 con 0x7f76f4100c50 2024-02-10T06:42:35.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f8f7f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419e170 con 0x7f76f41027f0 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 0x7f76f4196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 0x7f76f4196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:57064/0 (socket says 172.21.15.184:57064) 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 -- 172.21.15.184:0/2233924057 learned_addr learned my addr 172.21.15.184:0/2233924057 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f1d9b700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f4101e50 0x7f76f4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f2d9d700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f41027f0 0x7f76f419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 -- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f41027f0 msgr2=0x7f76f419d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f41027f0 0x7f76f419d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 -- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f4101e50 msgr2=0x7f76f4196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f4101e50 0x7f76f4196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 -- 172.21.15.184:0/2233924057 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80095f0 con 0x7f76f4100c50 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76f259c700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 0x7f76f4196310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c970 tx=0x7f76e400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4004220 con 0x7f76f4100c50 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.378+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f76e4007d50 con 0x7f76f4100c50 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.379+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f419e450 con 0x7f76f4100c50 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.379+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003c90 con 0x7f76f4100c50 2024-02-10T06:42:35.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.379+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f419e9a0 con 0x7f76f4100c50 2024-02-10T06:42:35.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.379+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f4107830 con 0x7f76f4100c50 2024-02-10T06:42:35.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.380+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f76e40078a0 con 0x7f76f4100c50 2024-02-10T06:42:35.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.380+0000 7f76e37fe700 1 --2- 172.21.15.184:0/2233924057 >> v2:172.21.15.174:6800/2183136494 conn(0x7f76dc06d990 0x7f76dc06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:42:35.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.381+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3205+0+0 (secure 0 0 0) 0x7f76e408c1f0 con 0x7f76f4100c50 2024-02-10T06:42:35.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.381+0000 7f76f1d9b700 1 --2- 172.21.15.184:0/2233924057 >> v2:172.21.15.174:6800/2183136494 conn(0x7f76dc06d990 0x7f76dc06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:42:35.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.381+0000 7f76f1d9b700 1 --2- 172.21.15.184:0/2233924057 >> v2:172.21.15.174:6800/2183136494 conn(0x7f76dc06d990 0x7f76dc06fe50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76e8000c00 tx=0x7f76e8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:42:35.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.385+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76e4056750 con 0x7f76f4100c50 2024-02-10T06:42:35.597 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:35.596+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f76f4062160 con 0x7f76dc06d990 2024-02-10T06:42:36.513 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:36 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:42:36.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:36 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:42:36.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:36 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:36 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:42:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:36 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:42:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:36 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:36.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:36 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:42:36.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:36 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:42:36.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:36 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:37.502 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:37 smithi184 ceph-mon[33513]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:42:37.502 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:37 smithi184 ceph-mon[33513]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-10T06:42:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:37 smithi174 ceph-mon[29878]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:42:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:37 smithi174 ceph-mon[29878]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-10T06:42:37.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:37 smithi174 ceph-mon[39523]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:42:37.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:37 smithi174 ceph-mon[39523]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-10T06:42:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/2385559691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]: dispatch 2024-02-10T06:42:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]: dispatch 2024-02-10T06:42:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]': finished 2024-02-10T06:42:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:38 smithi184 ceph-mon[33513]: osdmap e35: 6 total, 5 up, 6 in 2024-02-10T06:42:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:38 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:38 smithi184 ceph-mon[33513]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/2385559691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]: dispatch 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]: dispatch 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]': finished 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[29878]: osdmap e35: 6 total, 5 up, 6 in 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[29878]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T06:42:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/2385559691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]: dispatch 2024-02-10T06:42:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]: dispatch 2024-02-10T06:42:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21af74d5-65b4-4cfe-b5be-68d3ba4516b4"}]': finished 2024-02-10T06:42:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[39523]: osdmap e35: 6 total, 5 up, 6 in 2024-02-10T06:42:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:38 smithi174 ceph-mon[39523]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T06:42:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/4019636444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:42:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/4019636444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:42:40.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/4019636444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:42:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:40 smithi184 ceph-mon[33513]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-10T06:42:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:40 smithi174 ceph-mon[29878]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-10T06:42:41.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:40 smithi174 ceph-mon[39523]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-10T06:42:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:43 smithi184 ceph-mon[33513]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-10T06:42:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:43 smithi174 ceph-mon[29878]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-10T06:42:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:43 smithi174 ceph-mon[39523]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-10T06:42:45.699 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:45 smithi184 ceph-mon[33513]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T06:42:45.700 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T06:42:45.700 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:45 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:45 smithi174 ceph-mon[29878]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T06:42:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T06:42:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:45 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:45.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:45 smithi174 ceph-mon[39523]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T06:42:45.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T06:42:45.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:45 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:42:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:46 smithi174 ceph-mon[29878]: Deploying daemon osd.5 on smithi184 2024-02-10T06:42:46.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:46 smithi174 ceph-mon[39523]: Deploying daemon osd.5 on smithi184 2024-02-10T06:42:46.773 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:46 smithi184 ceph-mon[33513]: Deploying daemon osd.5 on smithi184 2024-02-10T06:42:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:47 smithi174 ceph-mon[29878]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:47 smithi174 ceph-mon[39523]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:47.751 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:47 smithi184 ceph-mon[33513]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:48 smithi184 ceph-mon[33513]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:48 smithi174 ceph-mon[29878]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:48 smithi174 ceph-mon[39523]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[29878]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[39523]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:51 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:51 smithi184 ceph-mon[33513]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:51 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:51 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:51 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:42:53.655 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:53 smithi184 ceph-mon[33513]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:53 smithi174 ceph-mon[29878]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:53 smithi174 ceph-mon[39523]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:53.966 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.965+0000 7f76e37fe700 1 -- 172.21.15.184:0/2233924057 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f76f4062160 con 0x7f76dc06d990 2024-02-10T06:42:53.966 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-02-10T06:42:53.968 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 >> v2:172.21.15.174:6800/2183136494 conn(0x7f76dc06d990 msgr2=0x7f76dc06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2233924057 >> v2:172.21.15.174:6800/2183136494 conn(0x7f76dc06d990 0x7f76dc06fe50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76e8000c00 tx=0x7f76e8009f10 comp rx=0 tx=0).stop 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 msgr2=0x7f76f4196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 0x7f76f4196310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76e400c970 tx=0x7f76e400cc80 comp rx=0 tx=0).stop 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 shutdown_connections 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f41027f0 0x7f76f419d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2233924057 >> v2:172.21.15.174:6800/2183136494 conn(0x7f76dc06d990 0x7f76dc06fe50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76f4100c50 0x7f76f4196310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 --2- 172.21.15.184:0/2233924057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f4101e50 0x7f76f4196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 >> 172.21.15.184:0/2233924057 conn(0x7f76f40fc1f0 msgr2=0x7f76f40fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.967+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 shutdown_connections 2024-02-10T06:42:53.969 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:42:53.968+0000 7f76f8f7f700 1 -- 172.21.15.184:0/2233924057 wait complete. 2024-02-10T06:42:54.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:54 smithi184 ceph-mon[33513]: from='osd.5 v2:172.21.15.184:6804/2462414067' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T06:42:54.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:54 smithi184 ceph-mon[33513]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T06:42:54.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:54 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:54.567 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:54 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:54.594 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.5.service 2024-02-10T06:42:54.596 INFO:tasks.cephadm:Deploying osd.6 on smithi184 with /dev/vg_nvme/lv_2... 2024-02-10T06:42:54.596 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-10T06:42:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[29878]: from='osd.5 v2:172.21.15.184:6804/2462414067' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T06:42:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[29878]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T06:42:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[39523]: from='osd.5 v2:172.21.15.184:6804/2462414067' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T06:42:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[39523]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T06:42:54.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:54.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:54 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:54.879 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: osdmap e36: 6 total, 5 up, 6 in 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='osd.5 v2:172.21.15.184:6804/2462414067' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:42:55.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: osdmap e37: 6 total, 5 up, 6 in 2024-02-10T06:42:55.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T06:42:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: osdmap e36: 6 total, 5 up, 6 in 2024-02-10T06:42:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='osd.5 v2:172.21.15.184:6804/2462414067' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: osdmap e37: 6 total, 5 up, 6 in 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: osdmap e36: 6 total, 5 up, 6 in 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='osd.5 v2:172.21.15.184:6804/2462414067' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: osdmap e37: 6 total, 5 up, 6 in 2024-02-10T06:42:55.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:55 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:56.123 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T06:42:55.904+0000 7f6ff82f7700 -1 osd.5 0 waiting for initial osdmap 2024-02-10T06:42:56.123 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 06:42:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T06:42:55.910+0000 7f6ff00e2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:42:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:56 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:56 smithi184 ceph-mon[33513]: from='osd.5 ' entity='osd.5' 2024-02-10T06:42:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:56 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:56 smithi174 ceph-mon[29878]: from='osd.5 ' entity='osd.5' 2024-02-10T06:42:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:56 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:56 smithi174 ceph-mon[39523]: from='osd.5 ' entity='osd.5' 2024-02-10T06:42:57.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:42:57.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:42:57.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:57.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:57.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: osd.5 v2:172.21.15.184:6804/2462414067 boot 2024-02-10T06:42:57.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: osdmap e38: 6 total, 6 up, 6 in 2024-02-10T06:42:57.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:57 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: osd.5 v2:172.21.15.184:6804/2462414067 boot 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: osdmap e38: 6 total, 6 up, 6 in 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:42:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:42:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-10T06:42:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: osd.5 v2:172.21.15.184:6804/2462414067 boot 2024-02-10T06:42:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: osdmap e38: 6 total, 6 up, 6 in 2024-02-10T06:42:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:57 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:42:58.291 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-10T06:42:58.292 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-10T06:42:58.292 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-10T06:42:58.292 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-10T06:42:58.292 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123936 s, 846 MB/s 2024-02-10T06:42:58.292 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-10T06:42:58.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:58 smithi184 ceph-mon[33513]: osdmap e39: 6 total, 6 up, 6 in 2024-02-10T06:42:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:58 smithi174 ceph-mon[29878]: osdmap e39: 6 total, 6 up, 6 in 2024-02-10T06:42:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:58 smithi174 ceph-mon[39523]: osdmap e39: 6 total, 6 up, 6 in 2024-02-10T06:42:59.219 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-02-10T06:42:59.647 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:42:59.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:59 smithi174 ceph-mon[29878]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:42:59.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:42:59 smithi174 ceph-mon[29878]: osdmap e40: 6 total, 6 up, 6 in 2024-02-10T06:42:59.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:59 smithi174 ceph-mon[39523]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:42:59.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:42:59 smithi174 ceph-mon[39523]: osdmap e40: 6 total, 6 up, 6 in 2024-02-10T06:42:59.769 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:59 smithi184 ceph-mon[33513]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:42:59.769 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:42:59 smithi184 ceph-mon[33513]: osdmap e40: 6 total, 6 up, 6 in 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: osdmap e41: 6 total, 6 up, 6 in 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.795 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.796 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:00 smithi184 ceph-mon[33513]: osdmap e42: 6 total, 6 up, 6 in 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: osdmap e41: 6 total, 6 up, 6 in 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[29878]: osdmap e42: 6 total, 6 up, 6 in 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: osdmap e41: 6 total, 6 up, 6 in 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:00.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:00.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:00 smithi174 ceph-mon[39523]: osdmap e42: 6 total, 6 up, 6 in 2024-02-10T06:43:01.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 -- 172.21.15.184:0/4197168015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a481027f0 msgr2=0x7f9a481090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:01.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 --2- 172.21.15.184:0/4197168015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a481027f0 0x7f9a481090a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b0d0 tx=0x7f9a4000b3e0 comp rx=0 tx=0).stop 2024-02-10T06:43:01.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 -- 172.21.15.184:0/4197168015 shutdown_connections 2024-02-10T06:43:01.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 --2- 172.21.15.184:0/4197168015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a481027f0 0x7f9a481090a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:01.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 --2- 172.21.15.184:0/4197168015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a48100c50 0x7f9a48101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:01.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 --2- 172.21.15.184:0/4197168015 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a48101e50 0x7f9a481022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:01.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 -- 172.21.15.184:0/4197168015 >> 172.21.15.184:0/4197168015 conn(0x7f9a480fc1f0 msgr2=0x7f9a480fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:01.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 -- 172.21.15.184:0/4197168015 shutdown_connections 2024-02-10T06:43:01.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.304+0000 7f9a4d756700 1 -- 172.21.15.184:0/4197168015 wait complete. 2024-02-10T06:43:01.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 Processor -- start 2024-02-10T06:43:01.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 -- start start 2024-02-10T06:43:01.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a48100c50 0x7f9a481963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:01.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 0x7f9a481968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:01.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a481027f0 0x7f9a4819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:01.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4819df40 con 0x7f9a48100c50 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4819e080 con 0x7f9a48101e50 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a4d756700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4819e210 con 0x7f9a481027f0 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a467fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 0x7f9a481968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a467fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 0x7f9a481968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:43680/0 (socket says 172.21.15.184:43680) 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a467fc700 1 -- 172.21.15.184:0/458486014 learned_addr learned my addr 172.21.15.184:0/458486014 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.305+0000 7f9a46ffd700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a48100c50 0x7f9a481963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a477fe700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a481027f0 0x7f9a4819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a467fc700 1 -- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a481027f0 msgr2=0x7f9a4819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a467fc700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a481027f0 0x7f9a4819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a467fc700 1 -- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a48100c50 msgr2=0x7f9a481963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a467fc700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a48100c50 0x7f9a481963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:01.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a467fc700 1 -- 172.21.15.184:0/458486014 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40009d00 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a467fc700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 0x7f9a481968f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00fb60 tx=0x7f9a3c00fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c009b40 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4806a420 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9a3c009ca0 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c018650 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.306+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4806a920 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.307+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4810c5a0 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.308+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f9a3c01a450 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.308+0000 7f9a2ffff700 1 --2- 172.21.15.184:0/458486014 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9a3006d990 0x7f9a3006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.308+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 3497+0+0 (secure 0 0 0) 0x7f9a3c08e620 con 0x7f9a48101e50 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.308+0000 7f9a46ffd700 1 --2- 172.21.15.184:0/458486014 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9a3006d990 0x7f9a3006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:01.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.309+0000 7f9a46ffd700 1 --2- 172.21.15.184:0/458486014 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9a3006d990 0x7f9a3006fe50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a48101cb0 tx=0x7f9a3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:01.313 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.313+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a3c0589e0 con 0x7f9a48101e50 2024-02-10T06:43:01.513 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:01.512+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a4806ab40 con 0x7f9a3006d990 2024-02-10T06:43:01.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:01 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi184 2024-02-10T06:43:01.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:01 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi184 to 8544M 2024-02-10T06:43:01.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:01 smithi184 ceph-mon[33513]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:01.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:01 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi184 2024-02-10T06:43:01.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:01 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi184 to 8544M 2024-02-10T06:43:01.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:01 smithi174 ceph-mon[29878]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:01.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:01 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi184 2024-02-10T06:43:01.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:01 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi184 to 8544M 2024-02-10T06:43:01.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:01 smithi174 ceph-mon[39523]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:02.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:02 smithi184 ceph-mon[33513]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:43:02.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:43:02.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:43:02.871 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:02 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[29878]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[39523]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:43:02.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:43:02.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:02 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:03 smithi184 ceph-mon[33513]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:03.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:03 smithi174 ceph-mon[29878]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:03.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:03 smithi174 ceph-mon[39523]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/502418399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]: dispatch 2024-02-10T06:43:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]: dispatch 2024-02-10T06:43:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]': finished 2024-02-10T06:43:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:04 smithi184 ceph-mon[33513]: osdmap e43: 7 total, 6 up, 7 in 2024-02-10T06:43:04.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/502418399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]: dispatch 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]: dispatch 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]': finished 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[29878]: osdmap e43: 7 total, 6 up, 7 in 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/502418399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]: dispatch 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]: dispatch 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffc0976a-c58c-4798-9c02-a1d7b828b46e"}]': finished 2024-02-10T06:43:04.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[39523]: osdmap e43: 7 total, 6 up, 7 in 2024-02-10T06:43:04.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:05 smithi184 ceph-mon[33513]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/1557590953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:43:05.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:05 smithi174 ceph-mon[29878]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:05.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/1557590953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:43:05.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:05 smithi174 ceph-mon[39523]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:05.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/1557590953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:43:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:07 smithi184 ceph-mon[33513]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:07.988 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:07 smithi174 ceph-mon[29878]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:07.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:07 smithi174 ceph-mon[39523]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:08 smithi184 ceph-mon[33513]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:08 smithi174 ceph-mon[29878]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:08 smithi174 ceph-mon[39523]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:11 smithi174 ceph-mon[29878]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T06:43:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:11 smithi174 ceph-mon[39523]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T06:43:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:11.744 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:11 smithi184 ceph-mon[33513]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:11.744 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T06:43:11.744 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:12.609 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:12 smithi184 ceph-mon[33513]: Deploying daemon osd.6 on smithi184 2024-02-10T06:43:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:12 smithi174 ceph-mon[29878]: Deploying daemon osd.6 on smithi184 2024-02-10T06:43:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:12 smithi174 ceph-mon[39523]: Deploying daemon osd.6 on smithi184 2024-02-10T06:43:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:13 smithi184 ceph-mon[33513]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:13 smithi174 ceph-mon[29878]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:13 smithi174 ceph-mon[39523]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:15.734 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:15 smithi184 ceph-mon[33513]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:15.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:15 smithi174 ceph-mon[29878]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:15 smithi174 ceph-mon[39523]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:17.666 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:17 smithi184 ceph-mon[33513]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:17.666 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:17.666 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:17.666 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:17 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[29878]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[39523]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:17 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:18.938 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:18 smithi184 ceph-mon[33513]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:18 smithi174 ceph-mon[29878]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:18 smithi174 ceph-mon[39523]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:20.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.428+0000 7f9a2ffff700 1 -- 172.21.15.184:0/458486014 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9a4806ab40 con 0x7f9a3006d990 2024-02-10T06:43:20.430 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 6 on host 'smithi184' 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9a3006d990 msgr2=0x7f9a3006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 --2- 172.21.15.184:0/458486014 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9a3006d990 0x7f9a3006fe50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a48101cb0 tx=0x7f9a3800b410 comp rx=0 tx=0).stop 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 msgr2=0x7f9a481968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 0x7f9a481968f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00fb60 tx=0x7f9a3c00fe70 comp rx=0 tx=0).stop 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 shutdown_connections 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a481027f0 0x7f9a4819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a48100c50 0x7f9a481963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 --2- 172.21.15.184:0/458486014 >> v2:172.21.15.174:6800/2183136494 conn(0x7f9a3006d990 0x7f9a3006fe50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 --2- 172.21.15.184:0/458486014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a48101e50 0x7f9a481968f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:20.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 >> 172.21.15.184:0/458486014 conn(0x7f9a480fc1f0 msgr2=0x7f9a480fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:20.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 shutdown_connections 2024-02-10T06:43:20.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:20.431+0000 7f9a4d756700 1 -- 172.21.15.184:0/458486014 wait complete. 2024-02-10T06:43:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='osd.6 v2:172.21.15.184:6808/2589400030' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='osd.6 v2:172.21.15.184:6808/2589400030' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T06:43:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T06:43:20.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:20.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:20.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='osd.6 v2:172.21.15.184:6808/2589400030' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:21.272 DEBUG:teuthology.orchestra.run.smithi184:osd.6> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.6.service 2024-02-10T06:43:21.275 INFO:tasks.cephadm:Deploying osd.7 on smithi184 with /dev/vg_nvme/lv_1... 2024-02-10T06:43:21.275 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-10T06:43:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:21 smithi184 ceph-mon[33513]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T06:43:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:21 smithi184 ceph-mon[33513]: osdmap e44: 7 total, 6 up, 7 in 2024-02-10T06:43:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:21 smithi184 ceph-mon[33513]: from='osd.6 v2:172.21.15.184:6808/2589400030' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:21 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:21 smithi184 ceph-mon[33513]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[29878]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[29878]: osdmap e44: 7 total, 6 up, 7 in 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[29878]: from='osd.6 v2:172.21.15.184:6808/2589400030' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[29878]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[39523]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T06:43:21.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[39523]: osdmap e44: 7 total, 6 up, 7 in 2024-02-10T06:43:21.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[39523]: from='osd.6 v2:172.21.15.184:6808/2589400030' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:21.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:21.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:21 smithi174 ceph-mon[39523]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:22.039 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:43:22.373 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T06:43:22.115+0000 7f37a65ac700 -1 osd.6 0 waiting for initial osdmap 2024-02-10T06:43:22.373 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T06:43:22.121+0000 7f379e397700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:43:22.746 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:43:22.746 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: osdmap e45: 7 total, 6 up, 7 in 2024-02-10T06:43:22.746 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.746 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.747 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: from='osd.6 ' entity='osd.6' 2024-02-10T06:43:22.747 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:22.747 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:43:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: osdmap e45: 7 total, 6 up, 7 in 2024-02-10T06:43:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: from='osd.6 ' entity='osd.6' 2024-02-10T06:43:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: osdmap e45: 7 total, 6 up, 7 in 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: from='osd.6 ' entity='osd.6' 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-10T06:43:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:43:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:43:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[29878]: osd.6 v2:172.21.15.184:6808/2589400030 boot 2024-02-10T06:43:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[29878]: osdmap e46: 7 total, 7 up, 7 in 2024-02-10T06:43:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[29878]: osdmap e47: 7 total, 7 up, 7 in 2024-02-10T06:43:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:43:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:43:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[39523]: osd.6 v2:172.21.15.184:6808/2589400030 boot 2024-02-10T06:43:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[39523]: osdmap e46: 7 total, 7 up, 7 in 2024-02-10T06:43:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:23 smithi174 ceph-mon[39523]: osdmap e47: 7 total, 7 up, 7 in 2024-02-10T06:43:24.059 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:23 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:43:24.059 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:23 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:43:24.059 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:23 smithi184 ceph-mon[33513]: osd.6 v2:172.21.15.184:6808/2589400030 boot 2024-02-10T06:43:24.059 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:23 smithi184 ceph-mon[33513]: osdmap e46: 7 total, 7 up, 7 in 2024-02-10T06:43:24.059 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:43:24.059 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:23 smithi184 ceph-mon[33513]: osdmap e47: 7 total, 7 up, 7 in 2024-02-10T06:43:25.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:24 smithi184 ceph-mon[33513]: osdmap e48: 7 total, 7 up, 7 in 2024-02-10T06:43:25.118 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:24 smithi184 ceph-mon[33513]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:24 smithi174 ceph-mon[29878]: osdmap e48: 7 total, 7 up, 7 in 2024-02-10T06:43:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:24 smithi174 ceph-mon[29878]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:24 smithi174 ceph-mon[39523]: osdmap e48: 7 total, 7 up, 7 in 2024-02-10T06:43:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:24 smithi174 ceph-mon[39523]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:25.559 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-10T06:43:25.559 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-10T06:43:25.559 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-02-10T06:43:25.559 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-02-10T06:43:25.559 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113841 s, 921 MB/s 2024-02-10T06:43:25.559 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: osdmap e49: 7 total, 7 up, 7 in 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi184 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi184 to 5696M 2024-02-10T06:43:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:26.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:26.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:25 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: osdmap e49: 7 total, 7 up, 7 in 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi184 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi184 to 5696M 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:26.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: osdmap e49: 7 total, 7 up, 7 in 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi184 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi184 to 5696M 2024-02-10T06:43:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:25 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:26.382 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-02-10T06:43:26.617 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:43:27.016 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:26 smithi184 ceph-mon[33513]: osdmap e50: 7 total, 7 up, 7 in 2024-02-10T06:43:27.016 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:26 smithi184 ceph-mon[33513]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-02-10T06:43:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:26 smithi174 ceph-mon[29878]: osdmap e50: 7 total, 7 up, 7 in 2024-02-10T06:43:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:26 smithi174 ceph-mon[29878]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-02-10T06:43:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:26 smithi174 ceph-mon[39523]: osdmap e50: 7 total, 7 up, 7 in 2024-02-10T06:43:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:26 smithi174 ceph-mon[39523]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 -- 172.21.15.184:0/1638089438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca01038e0 msgr2=0x7f2ca010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 --2- 172.21.15.184:0/1638089438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca01038e0 0x7f2ca010a190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b0d0 tx=0x7f2c9800b3e0 comp rx=0 tx=0).stop 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 -- 172.21.15.184:0/1638089438 shutdown_connections 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 --2- 172.21.15.184:0/1638089438 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca01038e0 0x7f2ca010a190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 --2- 172.21.15.184:0/1638089438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ca0101d40 0x7f2ca0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 --2- 172.21.15.184:0/1638089438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ca0102f40 0x7f2ca01033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 -- 172.21.15.184:0/1638089438 >> 172.21.15.184:0/1638089438 conn(0x7f2ca00fd300 msgr2=0x7f2ca00ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:28.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.316+0000 7f2ca8324700 1 -- 172.21.15.184:0/1638089438 shutdown_connections 2024-02-10T06:43:28.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.317+0000 7f2ca8324700 1 -- 172.21.15.184:0/1638089438 wait complete. 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.317+0000 7f2ca8324700 1 Processor -- start 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.317+0000 7f2ca8324700 1 -- start start 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ca0101d40 0x7f2ca01974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 0x7f2ca0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ca01038e0 0x7f2ca019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca019f080 con 0x7f2ca0101d40 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca019f1c0 con 0x7f2ca0102f40 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca019f350 con 0x7f2ca01038e0 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 0x7f2ca0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:28.321 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 0x7f2ca0197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:53370/0 (socket says 172.21.15.184:53370) 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 -- 172.21.15.184:0/2256230023 learned_addr learned my addr 172.21.15.184:0/2256230023 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca60c0700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ca0101d40 0x7f2ca01974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca68c1700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ca01038e0 0x7f2ca019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 -- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ca01038e0 msgr2=0x7f2ca019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ca01038e0 0x7f2ca019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 -- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ca0101d40 msgr2=0x7f2ca01974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ca0101d40 0x7f2ca01974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 -- 172.21.15.184:0/2256230023 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98009d00 con 0x7f2ca0102f40 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca58bf700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 0x7f2ca0197a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000d950 tx=0x7f2c9000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000f890 con 0x7f2ca0102f40 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2c90010460 con 0x7f2ca0102f40 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca019f630 con 0x7f2ca0102f40 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000fea0 con 0x7f2ca0102f40 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.318+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca019fb30 con 0x7f2ca0102f40 2024-02-10T06:43:28.322 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.319+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca0108920 con 0x7f2ca0102f40 2024-02-10T06:43:28.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.320+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f2c9000f9f0 con 0x7f2ca0102f40 2024-02-10T06:43:28.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.320+0000 7f2c977fe700 1 --2- 172.21.15.184:0/2256230023 >> v2:172.21.15.174:6800/2183136494 conn(0x7f2c8c06d990 0x7f2c8c06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:28.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.320+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 3789+0+0 (secure 0 0 0) 0x7f2c9008c640 con 0x7f2ca0102f40 2024-02-10T06:43:28.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.321+0000 7f2ca60c0700 1 --2- 172.21.15.184:0/2256230023 >> v2:172.21.15.174:6800/2183136494 conn(0x7f2c8c06d990 0x7f2c8c06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:28.323 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.321+0000 7f2ca60c0700 1 --2- 172.21.15.184:0/2256230023 >> v2:172.21.15.174:6800/2183136494 conn(0x7f2c8c06d990 0x7f2c8c06fe50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0102da0 tx=0x7f2c9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:28.325 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.325+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c900568e0 con 0x7f2ca0102f40 2024-02-10T06:43:28.526 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:28.525+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ca010d0c0 con 0x7f2c8c06d990 2024-02-10T06:43:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:28 smithi184 ceph-mon[33513]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-02-10T06:43:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:28 smithi184 ceph-mon[33513]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:43:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:43:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:43:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:28 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[29878]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-02-10T06:43:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[29878]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:43:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:43:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:43:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[39523]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-02-10T06:43:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[39523]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:43:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T06:43:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T06:43:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:28 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[39523]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-02-10T06:43:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/1257769760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]: dispatch 2024-02-10T06:43:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]: dispatch 2024-02-10T06:43:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]': finished 2024-02-10T06:43:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[39523]: osdmap e51: 8 total, 7 up, 8 in 2024-02-10T06:43:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[29878]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/1257769760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]: dispatch 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]: dispatch 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]': finished 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[29878]: osdmap e51: 8 total, 7 up, 8 in 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:31 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:31.740 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:31 smithi184 ceph-mon[33513]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-02-10T06:43:31.741 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/1257769760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]: dispatch 2024-02-10T06:43:31.741 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]: dispatch 2024-02-10T06:43:31.741 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a"}]': finished 2024-02-10T06:43:31.741 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:31 smithi184 ceph-mon[33513]: osdmap e51: 8 total, 7 up, 8 in 2024-02-10T06:43:31.741 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:31 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:32.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/1536802707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:43:32.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/1536802707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:43:32.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/1536802707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T06:43:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:33 smithi174 ceph-mon[29878]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:33 smithi174 ceph-mon[39523]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:33.774 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:33 smithi184 ceph-mon[33513]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:35.694 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:35 smithi184 ceph-mon[33513]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:35 smithi174 ceph-mon[29878]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:35.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:35 smithi174 ceph-mon[39523]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:37.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:37 smithi184 ceph-mon[33513]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:37 smithi174 ceph-mon[29878]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:37.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:37 smithi174 ceph-mon[39523]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:39.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:38 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T06:43:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:38 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:38 smithi184 ceph-mon[33513]: Deploying daemon osd.7 on smithi184 2024-02-10T06:43:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:38 smithi184 ceph-mon[33513]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[29878]: Deploying daemon osd.7 on smithi184 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[29878]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[39523]: Deploying daemon osd.7 on smithi184 2024-02-10T06:43:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:38 smithi174 ceph-mon[39523]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:41 smithi184 ceph-mon[33513]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:41 smithi174 ceph-mon[29878]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:41 smithi174 ceph-mon[39523]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:43.564 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:43 smithi184 ceph-mon[33513]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:43 smithi174 ceph-mon[29878]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:43 smithi174 ceph-mon[39523]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:44.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:44.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:44 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:44 smithi184 ceph-mon[33513]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[29878]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:44.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:44 smithi174 ceph-mon[39523]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:47.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.391+0000 7f2c977fe700 1 -- 172.21.15.184:0/2256230023 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2ca010d0c0 con 0x7f2c8c06d990 2024-02-10T06:43:47.392 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 7 on host 'smithi184' 2024-02-10T06:43:47.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 >> v2:172.21.15.174:6800/2183136494 conn(0x7f2c8c06d990 msgr2=0x7f2c8c06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:47.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 --2- 172.21.15.184:0/2256230023 >> v2:172.21.15.174:6800/2183136494 conn(0x7f2c8c06d990 0x7f2c8c06fe50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0102da0 tx=0x7f2c9c00b410 comp rx=0 tx=0).stop 2024-02-10T06:43:47.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 msgr2=0x7f2ca0197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:47.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 0x7f2ca0197a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000d950 tx=0x7f2c9000dc60 comp rx=0 tx=0).stop 2024-02-10T06:43:47.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 shutdown_connections 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ca01038e0 0x7f2ca019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ca0101d40 0x7f2ca01974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 --2- 172.21.15.184:0/2256230023 >> v2:172.21.15.174:6800/2183136494 conn(0x7f2c8c06d990 0x7f2c8c06fe50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 --2- 172.21.15.184:0/2256230023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ca0102f40 0x7f2ca0197a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 >> 172.21.15.184:0/2256230023 conn(0x7f2ca00fd300 msgr2=0x7f2ca00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 shutdown_connections 2024-02-10T06:43:47.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:47.393+0000 7f2ca8324700 1 -- 172.21.15.184:0/2256230023 wait complete. 2024-02-10T06:43:47.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:47 smithi184 ceph-mon[33513]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:47 smithi184 ceph-mon[33513]: from='osd.7 v2:172.21.15.184:6812/3506789608' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T06:43:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:47 smithi184 ceph-mon[33513]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T06:43:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:47 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:47 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[29878]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[29878]: from='osd.7 v2:172.21.15.184:6812/3506789608' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T06:43:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[29878]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T06:43:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[39523]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[39523]: from='osd.7 v2:172.21.15.184:6812/3506789608' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T06:43:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[39523]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T06:43:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:47 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.048 DEBUG:teuthology.orchestra.run.smithi184:osd.7> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.7.service 2024-02-10T06:43:48.051 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-10T06:43:48.051 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd stat -f json 2024-02-10T06:43:48.303 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='osd.7 v2:172.21.15.184:6812/3506789608' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: osdmap e52: 8 total, 7 up, 8 in 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: osdmap e53: 8 total, 7 up, 8 in 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='osd.7 v2:172.21.15.184:6812/3506789608' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: osdmap e52: 8 total, 7 up, 8 in 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: osdmap e53: 8 total, 7 up, 8 in 2024-02-10T06:43:48.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.731 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:48 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.866 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T06:43:48.866 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='osd.7 v2:172.21.15.184:6812/3506789608' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:48.866 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: osdmap e52: 8 total, 7 up, 8 in 2024-02-10T06:43:48.866 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.866 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-02-10T06:43:48.867 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.867 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:48.867 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-02-10T06:43:48.867 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: osdmap e53: 8 total, 7 up, 8 in 2024-02-10T06:43:48.867 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:48.867 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:49.123 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T06:43:48.865+0000 7f27125bd700 -1 osd.7 0 waiting for initial osdmap 2024-02-10T06:43:49.123 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 06:43:48 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T06:43:48.870+0000 7f270a3a8700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T06:43:49.527 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:49 smithi184 ceph-mon[33513]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:49.528 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:49 smithi184 ceph-mon[33513]: from='osd.7 ' entity='osd.7' 2024-02-10T06:43:49.528 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:49 smithi184 ceph-mon[33513]: osd.7 v2:172.21.15.184:6812/3506789608 boot 2024-02-10T06:43:49.528 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:49 smithi184 ceph-mon[33513]: osdmap e54: 8 total, 8 up, 8 in 2024-02-10T06:43:49.528 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:49 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:49.732 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[29878]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:49.732 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[29878]: from='osd.7 ' entity='osd.7' 2024-02-10T06:43:49.732 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[29878]: osd.7 v2:172.21.15.184:6812/3506789608 boot 2024-02-10T06:43:49.732 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[29878]: osdmap e54: 8 total, 8 up, 8 in 2024-02-10T06:43:49.732 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:49.733 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[39523]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:49.733 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[39523]: from='osd.7 ' entity='osd.7' 2024-02-10T06:43:49.733 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[39523]: osd.7 v2:172.21.15.184:6812/3506789608 boot 2024-02-10T06:43:49.733 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[39523]: osdmap e54: 8 total, 8 up, 8 in 2024-02-10T06:43:49.733 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:49 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:43:50.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.120+0000 7f87d512c700 1 -- 172.21.15.174:0/476375576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0101d50 msgr2=0x7f87d0102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.120+0000 7f87d512c700 1 --2- 172.21.15.174:0/476375576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0101d50 0x7f87d0102170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87c00099c0 tx=0x7f87c0009cd0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 -- 172.21.15.174:0/476375576 shutdown_connections 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 --2- 172.21.15.174:0/476375576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d01038f0 0x7f87d010a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 --2- 172.21.15.174:0/476375576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0101d50 0x7f87d0102170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 --2- 172.21.15.174:0/476375576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d0102f50 0x7f87d01033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 -- 172.21.15.174:0/476375576 >> 172.21.15.174:0/476375576 conn(0x7f87d00fd330 msgr2=0x7f87d00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 -- 172.21.15.174:0/476375576 shutdown_connections 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 -- 172.21.15.174:0/476375576 wait complete. 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 Processor -- start 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.121+0000 7f87d512c700 1 -- start start 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 0x7f87d0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 0x7f87d0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:50.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d01038f0 0x7f87d019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d019efc0 con 0x7f87d0102f50 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d019f100 con 0x7f87d01038f0 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d019f290 con 0x7f87d0101d50 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ced9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 0x7f87d0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 0x7f87d0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ced9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 0x7f87d0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:42822/0 (socket says 172.21.15.174:42822) 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 0x7f87d0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:36690/0 (socket says 172.21.15.174:36690) 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ced9d700 1 -- 172.21.15.174:0/3889732693 learned_addr learned my addr 172.21.15.174:0/3889732693 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87cf59e700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d01038f0 0x7f87d019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 -- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 msgr2=0x7f87d0197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 0x7f87d0197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 -- 172.21.15.174:0/3889732693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d01038f0 msgr2=0x7f87d019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d01038f0 0x7f87d019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.125 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 -- 172.21.15.174:0/3889732693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8009c40 con 0x7f87d0102f50 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ced9d700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 0x7f87d0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87ce59c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 0x7f87d0197970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87c400d980 tx=0x7f87c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4009930 con 0x7f87d0102f50 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.122+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c00095f0 con 0x7f87d0102f50 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.123+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f87c400be70 con 0x7f87d0102f50 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.123+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4010e80 con 0x7f87d0102f50 2024-02-10T06:43:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.123+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d019f930 con 0x7f87d0102f50 2024-02-10T06:43:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.124+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d0108930 con 0x7f87d0102f50 2024-02-10T06:43:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.124+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f87c400f3c0 con 0x7f87d0102f50 2024-02-10T06:43:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.124+0000 7f87b7fff700 1 --2- 172.21.15.174:0/3889732693 >> v2:172.21.15.174:6800/2183136494 conn(0x7f87b806d990 0x7f87b806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.124+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4114+0+0 (secure 0 0 0) 0x7f87c408c580 con 0x7f87d0102f50 2024-02-10T06:43:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.124+0000 7f87ced9d700 1 --2- 172.21.15.174:0/3889732693 >> v2:172.21.15.174:6800/2183136494 conn(0x7f87b806d990 0x7f87b806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.125+0000 7f87ced9d700 1 --2- 172.21.15.174:0/3889732693 >> v2:172.21.15.174:6800/2183136494 conn(0x7f87b806d990 0x7f87b806fe50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87c00095c0 tx=0x7f87c0009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:50.130 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.129+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f87c40566d0 con 0x7f87d0102f50 2024-02-10T06:43:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.321+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f87d010cd50 con 0x7f87d0102f50 2024-02-10T06:43:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.321+0000 7f87b7fff700 1 -- 172.21.15.174:0/3889732693 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v54) v1 ==== 74+0+130 (secure 0 0 0) 0x7f87c4059d90 con 0x7f87d0102f50 2024-02-10T06:43:50.322 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 >> v2:172.21.15.174:6800/2183136494 conn(0x7f87b806d990 msgr2=0x7f87b806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 --2- 172.21.15.174:0/3889732693 >> v2:172.21.15.174:6800/2183136494 conn(0x7f87b806d990 0x7f87b806fe50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87c00095c0 tx=0x7f87c0009510 comp rx=0 tx=0).stop 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 msgr2=0x7f87d0197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 0x7f87d0197970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f87c400d980 tx=0x7f87c400dc90 comp rx=0 tx=0).stop 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 shutdown_connections 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 --2- 172.21.15.174:0/3889732693 >> v2:172.21.15.174:6800/2183136494 conn(0x7f87b806d990 0x7f87b806fe50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87d01038f0 0x7f87d019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87d0101d50 0x7f87d0197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 --2- 172.21.15.174:0/3889732693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0102f50 0x7f87d0197970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 >> 172.21.15.174:0/3889732693 conn(0x7f87d00fd330 msgr2=0x7f87d00fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 shutdown_connections 2024-02-10T06:43:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:50.324+0000 7f87d512c700 1 -- 172.21.15.174:0/3889732693 wait complete. 2024-02-10T06:43:50.543 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:50 smithi184 ceph-mon[33513]: purged_snaps scrub starts 2024-02-10T06:43:50.543 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:50 smithi184 ceph-mon[33513]: purged_snaps scrub ok 2024-02-10T06:43:50.543 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3889732693' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T06:43:50.543 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:50 smithi184 ceph-mon[33513]: osdmap e55: 8 total, 8 up, 8 in 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[29878]: purged_snaps scrub starts 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[29878]: purged_snaps scrub ok 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3889732693' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[29878]: osdmap e55: 8 total, 8 up, 8 in 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[39523]: purged_snaps scrub starts 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[39523]: purged_snaps scrub ok 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3889732693' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T06:43:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:50 smithi174 ceph-mon[39523]: osdmap e55: 8 total, 8 up, 8 in 2024-02-10T06:43:51.197 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1707547429,"num_in_osds":8,"osd_in_since":1707547411,"num_remapped_pgs":1} 2024-02-10T06:43:51.198 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T06:43:51.439 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:43:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:51 smithi184 ceph-mon[33513]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:51 smithi184 ceph-mon[33513]: osdmap e56: 8 total, 8 up, 8 in 2024-02-10T06:43:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:51 smithi174 ceph-mon[29878]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:51 smithi174 ceph-mon[29878]: osdmap e56: 8 total, 8 up, 8 in 2024-02-10T06:43:51.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:51 smithi174 ceph-mon[39523]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-10T06:43:51.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:51 smithi174 ceph-mon[39523]: osdmap e56: 8 total, 8 up, 8 in 2024-02-10T06:43:53.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 -- 172.21.15.174:0/2264669501 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208100eb0 msgr2=0x7f9208101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 --2- 172.21.15.174:0/2264669501 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208100eb0 0x7f9208101290 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009990 tx=0x7f91f8009ca0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 -- 172.21.15.174:0/2264669501 shutdown_connections 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 --2- 172.21.15.174:0/2264669501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 --2- 172.21.15.174:0/2264669501 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208100eb0 0x7f9208101290 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 --2- 172.21.15.174:0/2264669501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208101860 0x7f9208105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 -- 172.21.15.174:0/2264669501 >> 172.21.15.174:0/2264669501 conn(0x7f9208076c70 msgr2=0x7f9208077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 -- 172.21.15.174:0/2264669501 shutdown_connections 2024-02-10T06:43:53.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.145+0000 7f920c9f9700 1 -- 172.21.15.174:0/2264669501 wait complete. 2024-02-10T06:43:53.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 Processor -- start 2024-02-10T06:43:53.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 -- start start 2024-02-10T06:43:53.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208100eb0 0x7f92081965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:53.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208101860 0x7f9208196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:53.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92081973d0 con 0x7f9208105f00 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920819bb60 con 0x7f9208100eb0 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920c9f9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920819bd20 con 0x7f9208101860 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f9206d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208100eb0 0x7f92081965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f9206d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:36702/0 (socket says 172.21.15.174:36702) 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f9206d9d700 1 -- 172.21.15.174:0/1506576090 learned_addr learned my addr 172.21.15.174:0/1506576090 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f9205d9b700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208101860 0x7f9208196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920659c700 1 -- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208101860 msgr2=0x7f9208196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920659c700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208101860 0x7f9208196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920659c700 1 -- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 msgr2=0x7f920819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920659c700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f920659c700 1 -- 172.21.15.174:0/1506576090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200009c40 con 0x7f9208100eb0 2024-02-10T06:43:53.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f9205d9b700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208101860 0x7f9208196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.146+0000 7f9206d9d700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.147+0000 7f920659c700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208100eb0 0x7f92081965d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91f80059d0 tx=0x7f91f8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.147+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f801d070 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.147+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f80095f0 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.147+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f91f800b680 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.147+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920819c220 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.147+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8022e40 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.148+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e8005120 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.148+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f91f800fb10 con 0x7f9208100eb0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.149+0000 7f91f77fe700 1 --2- 172.21.15.174:0/1506576090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f91f006d990 0x7f91f006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.149+0000 7f9205d9b700 1 --2- 172.21.15.174:0/1506576090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f91f006d990 0x7f91f006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:53.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.149+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f91f808df50 con 0x7f9208100eb0 2024-02-10T06:43:53.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.149+0000 7f9205d9b700 1 --2- 172.21.15.174:0/1506576090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f91f006d990 0x7f91f006fe50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f92081008d0 tx=0x7f91fc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:53.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.153+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91f80580a0 con 0x7f9208100eb0 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: Detected new or changed devices on smithi184 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi184 to 4272M 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: osdmap e57: 8 total, 8 up, 8 in 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: Detected new or changed devices on smithi184 2024-02-10T06:43:53.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi184 to 4272M 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: osdmap e57: 8 total, 8 up, 8 in 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:53.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:53.242 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:53 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.350 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.350+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91e80056a0 con 0x7f9208100eb0 2024-02-10T06:43:53.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.351+0000 7f91f77fe700 1 -- 172.21.15.174:0/1506576090 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+11545 (secure 0 0 0) 0x7f91f805b760 con 0x7f9208100eb0 2024-02-10T06:43:53.352 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:43:53.352 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":57,"fsid":"df845a14-c7de-11ee-95b8-87774f69a715","created":"2024-02-10T06:38:25.649613+0000","modified":"2024-02-10T06:43:52.456065+0000","last_up_change":"2024-02-10T06:43:49.428578+0000","last_in_change":"2024-02-10T06:43:31.395302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T06:41:40.659218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b892cc18-df62-424f-8250-d301c822e8f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6801","nonce":4183537518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4183537518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4183537518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4183537518}]},"public_addr":"172.21.15.174:6801/4183537518","cluster_addr":"172.21.15.174:6802/4183537518","heartbeat_back_addr":"172.21.15.174:6804/4183537518","heartbeat_front_addr":"172.21.15.174:6803/4183537518","state":["exists","up"]},{"osd":1,"uuid":"aebb7103-2298-403b-8df6-e078cf073484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6805","nonce":640275884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":640275884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":640275884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":640275884}]},"public_addr":"172.21.15.174:6805/640275884","cluster_addr":"172.21.15.174:6806/640275884","heartbeat_back_addr":"172.21.15.174:6808/640275884","heartbeat_front_addr":"172.21.15.174:6807/640275884","state":["exists","up"]},{"osd":2,"uuid":"11f1e7b8-5292-4c92-8823-1e9387e69d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6809","nonce":2318790055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2318790055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2318790055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":2318790055}]},"public_addr":"172.21.15.174:6809/2318790055","cluster_addr":"172.21.15.174:6810/2318790055","heartbeat_back_addr":"172.21.15.174:6812/2318790055","heartbeat_front_addr":"172.21.15.174:6811/2318790055","state":["exists","up"]},{"osd":3,"uuid":"6db6715d-ad5c-42a4-b051-0d2fdd2fef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6813","nonce":2745280023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2745280023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2745280023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6815","nonce":2745280023}]},"public_addr":"172.21.15.174:6813/2745280023","cluster_addr":"172.21.15.174:6814/2745280023","heartbeat_back_addr":"172.21.15.174:6816/2745280023","heartbeat_front_addr":"172.21.15.174:6815/2745280023","state":["exists","up"]},{"osd":4,"uuid":"3df6af28-331e-466d-ba68-0c7db4e6a616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3815653813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":3815653813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":3815653813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3815653813}]},"public_addr":"172.21.15.184:6800/3815653813","cluster_addr":"172.21.15.184:6801/3815653813","heartbeat_back_addr":"172.21.15.184:6803/3815653813","heartbeat_front_addr":"172.21.15.184:6802/3815653813","state":["exists","up"]},{"osd":5,"uuid":"21af74d5-65b4-4cfe-b5be-68d3ba4516b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2462414067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2462414067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":2462414067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2462414067}]},"public_addr":"172.21.15.184:6804/2462414067","cluster_addr":"172.21.15.184:6805/2462414067","heartbeat_back_addr":"172.21.15.184:6807/2462414067","heartbeat_front_addr":"172.21.15.184:6806/2462414067","state":["exists","up"]},{"osd":6,"uuid":"ffc0976a-c58c-4798-9c02-a1d7b828b46e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2589400030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2589400030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":2589400030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2589400030}]},"public_addr":"172.21.15.184:6808/2589400030","cluster_addr":"172.21.15.184:6809/2589400030","heartbeat_back_addr":"172.21.15.184:6811/2589400030","heartbeat_front_addr":"172.21.15.184:6810/2589400030","state":["exists","up"]},{"osd":7,"uuid":"ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3506789608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":3506789608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":3506789608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3506789608}]},"public_addr":"172.21.15.184:6812/3506789608","cluster_addr":"172.21.15.184:6813/3506789608","heartbeat_back_addr":"172.21.15.184:6815/3506789608","heartbeat_front_addr":"172.21.15.184:6814/3506789608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:40:46.684618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:41:12.730284+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:41:38.522191+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:04.175541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:28.672368+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:54.571801+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:43:20.849260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:43:47.772410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:0/3154304474":"2024-02-11T06:39:22.324013+0000","172.21.15.174:0/3421212038":"2024-02-11T06:39:22.324013+0000","172.21.15.174:6800/2787410738":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/1784760840":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/3867738525":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/2793344305":"2024-02-11T06:39:22.324013+0000","172.21.15.174:0/4219955631":"2024-02-11T06:38:55.602697+0000","172.21.15.174:6800/1048754011":"2024-02-11T06:39:22.324013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f91f006d990 msgr2=0x7f91f006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 --2- 172.21.15.174:0/1506576090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f91f006d990 0x7f91f006fe50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f92081008d0 tx=0x7f91fc00a300 comp rx=0 tx=0).stop 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208100eb0 msgr2=0x7f92081965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208100eb0 0x7f92081965d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91f80059d0 tx=0x7f91f8004730 comp rx=0 tx=0).stop 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 shutdown_connections 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9208105f00 0x7f920819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 --2- 172.21.15.174:0/1506576090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f91f006d990 0x7f91f006fe50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9208100eb0 0x7f92081965d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.354 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 --2- 172.21.15.174:0/1506576090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9208101860 0x7f9208196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:53.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 >> 172.21.15.174:0/1506576090 conn(0x7f9208076c70 msgr2=0x7f9208104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:53.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 shutdown_connections 2024-02-10T06:43:53.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:53.353+0000 7f920c9f9700 1 -- 172.21.15.174:0/1506576090 wait complete. 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: Detected new or changed devices on smithi184 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi184 to 4272M 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: osdmap e57: 8 total, 8 up, 8 in 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:43:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:53 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:43:54.008 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-10T06:41:40.659218+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-10T06:43:54.008 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-10T06:43:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1506576090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:43:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:54 smithi174 ceph-mon[29878]: osdmap e58: 8 total, 8 up, 8 in 2024-02-10T06:43:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1506576090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:43:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:54 smithi174 ceph-mon[39523]: osdmap e58: 8 total, 8 up, 8 in 2024-02-10T06:43:54.244 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:43:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1506576090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:43:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:54 smithi184 ceph-mon[33513]: osdmap e58: 8 total, 8 up, 8 in 2024-02-10T06:43:55.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:55 smithi184 ceph-mon[33513]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:43:55.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:55 smithi174 ceph-mon[29878]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:43:55.488 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:55 smithi174 ceph-mon[39523]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:43:55.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 -- 172.21.15.174:0/4048058892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0102730 msgr2=0x7f84b010ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:55.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 --2- 172.21.15.174:0/4048058892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0102730 0x7f84b010ebd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b0a0 tx=0x7f84a800b3b0 comp rx=0 tx=0).stop 2024-02-10T06:43:55.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 -- 172.21.15.174:0/4048058892 shutdown_connections 2024-02-10T06:43:55.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 --2- 172.21.15.174:0/4048058892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0102730 0x7f84b010ebd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:55.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 --2- 172.21.15.174:0/4048058892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84b0107e20 0x7f84b0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 --2- 172.21.15.174:0/4048058892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0101d70 0x7f84b01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.856+0000 7f84b6617700 1 -- 172.21.15.174:0/4048058892 >> 172.21.15.174:0/4048058892 conn(0x7f84b0078e50 msgr2=0x7f84b0079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 -- 172.21.15.174:0/4048058892 shutdown_connections 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 -- 172.21.15.174:0/4048058892 wait complete. 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 Processor -- start 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 -- start start 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0101d70 0x7f84b019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84b0102730 0x7f84b019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 0x7f84b0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0197ef0 con 0x7f84b0107e20 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0198030 con 0x7f84b0102730 2024-02-10T06:43:55.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.857+0000 7f84b6617700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01981f0 con 0x7f84b0101d70 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 0x7f84b0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84affff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0101d70 0x7f84b019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 0x7f84b0197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:42258/0 (socket says 172.21.15.174:42258) 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 -- 172.21.15.174:0/2763522295 learned_addr learned my addr 172.21.15.174:0/2763522295 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84af7fe700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84b0102730 0x7f84b019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 -- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0101d70 msgr2=0x7f84b019d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0101d70 0x7f84b019d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 -- 172.21.15.174:0/2763522295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84b0102730 msgr2=0x7f84b019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84b0102730 0x7f84b019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 -- 172.21.15.174:0/2763522295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009520 con 0x7f84b0107e20 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b4bb4700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 0x7f84b0197890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b070 tx=0x7f84a8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800e040 con 0x7f84b0107e20 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a8009d00 con 0x7f84b0107e20 2024-02-10T06:43:55.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f84a8009480 con 0x7f84b0107e20 2024-02-10T06:43:55.862 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a80052c0 con 0x7f84b0107e20 2024-02-10T06:43:55.862 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.858+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b01a45a0 con 0x7f84b0107e20 2024-02-10T06:43:55.862 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.859+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b0103460 con 0x7f84b0107e20 2024-02-10T06:43:55.862 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.860+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f84a801d470 con 0x7f84b0107e20 2024-02-10T06:43:55.862 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.860+0000 7f84ad7fa700 1 --2- 172.21.15.174:0/2763522295 >> v2:172.21.15.174:6800/2183136494 conn(0x7f849806d990 0x7f849806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:55.863 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.860+0000 7f84affff700 1 --2- 172.21.15.174:0/2763522295 >> v2:172.21.15.174:6800/2183136494 conn(0x7f849806d990 0x7f849806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:55.863 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.860+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f84a808de90 con 0x7f84b0107e20 2024-02-10T06:43:55.863 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.861+0000 7f84affff700 1 --2- 172.21.15.174:0/2763522295 >> v2:172.21.15.174:6800/2183136494 conn(0x7f849806d990 0x7f849806fe50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84b01989d0 tx=0x7f84a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:55.865 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:55.865+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84a8058000 con 0x7f84b0107e20 2024-02-10T06:43:56.060 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.057+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84b0198dc0 con 0x7f84b0107e20 2024-02-10T06:43:56.060 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.058+0000 7f84ad7fa700 1 -- 172.21.15.174:0/2763522295 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84a801d720 con 0x7f84b0107e20 2024-02-10T06:43:56.060 INFO:teuthology.orchestra.run.smithi174.stdout:pg_num: 1 2024-02-10T06:43:56.061 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 >> v2:172.21.15.174:6800/2183136494 conn(0x7f849806d990 msgr2=0x7f849806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:56.061 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 --2- 172.21.15.174:0/2763522295 >> v2:172.21.15.174:6800/2183136494 conn(0x7f849806d990 0x7f849806fe50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84b01989d0 tx=0x7f84a0009190 comp rx=0 tx=0).stop 2024-02-10T06:43:56.061 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 msgr2=0x7f84b0197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:56.061 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 0x7f84b0197890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b070 tx=0x7f84a8004d00 comp rx=0 tx=0).stop 2024-02-10T06:43:56.061 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 shutdown_connections 2024-02-10T06:43:56.061 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b0107e20 0x7f84b0197890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:56.062 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84b0101d70 0x7f84b019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:56.062 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 --2- 172.21.15.174:0/2763522295 >> v2:172.21.15.174:6800/2183136494 conn(0x7f849806d990 0x7f849806fe50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:56.062 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 --2- 172.21.15.174:0/2763522295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84b0102730 0x7f84b019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:56.062 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 >> 172.21.15.174:0/2763522295 conn(0x7f84b0078e50 msgr2=0x7f84b0105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:56.062 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 shutdown_connections 2024-02-10T06:43:56.062 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:43:56.060+0000 7f84b6617700 1 -- 172.21.15.174:0/2763522295 wait complete. 2024-02-10T06:43:56.368 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2763522295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T06:43:56.369 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2763522295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T06:43:56.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2763522295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T06:43:56.712 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi174 2024-02-10T06:43:56.712 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi174=foo.a' 2024-02-10T06:43:56.974 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:43:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:57 smithi184 ceph-mon[33513]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T06:43:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:57 smithi174 ceph-mon[29878]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T06:43:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:57 smithi174 ceph-mon[39523]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 -- 172.21.15.184:0/3513723868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c103370 msgr2=0x7f3b5c103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 --2- 172.21.15.184:0/3513723868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c103370 0x7f3b5c103790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009990 tx=0x7f3b4c009ca0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 -- 172.21.15.184:0/3513723868 shutdown_connections 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 --2- 172.21.15.184:0/3513723868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 0x7f3b5c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 --2- 172.21.15.184:0/3513723868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c103370 0x7f3b5c103790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 --2- 172.21.15.184:0/3513723868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b5c0ffdf0 0x7f3b5c100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 -- 172.21.15.184:0/3513723868 >> 172.21.15.184:0/3513723868 conn(0x7f3b5c0fb9b0 msgr2=0x7f3b5c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:58.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 -- 172.21.15.184:0/3513723868 shutdown_connections 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.727+0000 7f3b61fee700 1 -- 172.21.15.184:0/3513723868 wait complete. 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 Processor -- start 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 -- start start 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 0x7f3b5c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 0x7f3b5c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b5c103370 0x7f3b5c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c19efb0 con 0x7f3b5c1007b0 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c19f0f0 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b61fee700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c19f280 con 0x7f3b5c103370 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b5b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 0x7f3b5c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b5b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 0x7f3b5c197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:34072/0 (socket says 172.21.15.184:34072) 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b5b7fe700 1 -- 172.21.15.184:0/108316002 learned_addr learned my addr 172.21.15.184:0/108316002 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.728+0000 7f3b5affd700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 0x7f3b5c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:58.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5bfff700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b5c103370 0x7f3b5c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5b7fe700 1 -- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b5c103370 msgr2=0x7f3b5c19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5b7fe700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b5c103370 0x7f3b5c19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5b7fe700 1 -- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 msgr2=0x7f3b5c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5b7fe700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 0x7f3b5c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5b7fe700 1 -- 172.21.15.184:0/108316002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c0095f0 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5affd700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 0x7f3b5c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b5b7fe700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 0x7f3b5c197470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c005b40 tx=0x7f3b4c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c01d070 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c19f500 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f3b4c021410 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c017650 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.732 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.729+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c19f920 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.733 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.730+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c108af0 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.733 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.731+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f3b4c00fb00 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.733 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.731+0000 7f3b58ff9700 1 --2- 172.21.15.184:0/108316002 >> v2:172.21.15.174:6800/2183136494 conn(0x7f3b4406d990 0x7f3b4406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:43:58.733 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.731+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f3b4c08dd50 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.734 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.731+0000 7f3b5affd700 1 --2- 172.21.15.184:0/108316002 >> v2:172.21.15.174:6800/2183136494 conn(0x7f3b4406d990 0x7f3b4406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:43:58.734 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.732+0000 7f3b5affd700 1 --2- 172.21.15.184:0/108316002 >> v2:172.21.15.174:6800/2183136494 conn(0x7f3b4406d990 0x7f3b4406fe50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c06a3f0 tx=0x7f3b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:43:58.736 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.736+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b4c091030 con 0x7f3b5c0ffdf0 2024-02-10T06:43:58.943 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.942+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi174=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b5c062160 con 0x7f3b4406d990 2024-02-10T06:43:58.951 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.950+0000 7f3b58ff9700 1 -- 172.21.15.184:0/108316002 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f3b5c062160 con 0x7f3b4406d990 2024-02-10T06:43:58.951 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled rgw.foo.a update... 2024-02-10T06:43:58.953 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 >> v2:172.21.15.174:6800/2183136494 conn(0x7f3b4406d990 msgr2=0x7f3b4406fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:58.953 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 --2- 172.21.15.184:0/108316002 >> v2:172.21.15.174:6800/2183136494 conn(0x7f3b4406d990 0x7f3b4406fe50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c06a3f0 tx=0x7f3b50009260 comp rx=0 tx=0).stop 2024-02-10T06:43:58.953 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 msgr2=0x7f3b5c197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:43:58.953 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 0x7f3b5c197470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c005b40 tx=0x7f3b4c004730 comp rx=0 tx=0).stop 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 shutdown_connections 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b5c103370 0x7f3b5c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 --2- 172.21.15.184:0/108316002 >> v2:172.21.15.174:6800/2183136494 conn(0x7f3b4406d990 0x7f3b4406fe50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b5c0ffdf0 0x7f3b5c197470 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 --2- 172.21.15.184:0/108316002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b5c1007b0 0x7f3b5c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 >> 172.21.15.184:0/108316002 conn(0x7f3b5c0fb9b0 msgr2=0x7f3b5c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 shutdown_connections 2024-02-10T06:43:58.954 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:43:58.952+0000 7f3b61fee700 1 -- 172.21.15.184:0/108316002 wait complete. 2024-02-10T06:43:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:58 smithi184 ceph-mon[33513]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-10T06:43:59.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:58 smithi174 ceph-mon[29878]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-10T06:43:59.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:58 smithi174 ceph-mon[39523]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-10T06:43:59.545 DEBUG:teuthology.orchestra.run.smithi174:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@rgw.foo.a.service 2024-02-10T06:43:59.547 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi184 2024-02-10T06:43:59.548 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-10T06:43:59.785 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi174=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: Saving service rgw.foo.a spec with placement smithi174=foo.a;count:1 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T06:44:00.160 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.161 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:00.161 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:43:59 smithi184 ceph-mon[33513]: Deploying daemon rgw.foo.a on smithi174 2024-02-10T06:44:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi174=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: Saving service rgw.foo.a spec with placement smithi174=foo.a;count:1 2024-02-10T06:44:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[29878]: Deploying daemon rgw.foo.a on smithi174 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi174=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: Saving service rgw.foo.a spec with placement smithi174=foo.a;count:1 2024-02-10T06:44:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:43:59 smithi174 ceph-mon[39523]: Deploying daemon rgw.foo.a on smithi174 2024-02-10T06:44:01.192 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:00 smithi184 ceph-mon[33513]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-10T06:44:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:00 smithi174 ceph-mon[29878]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-10T06:44:01.239 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 systemd[1]: Starting Ceph rgw.foo.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:44:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:00 smithi174 ceph-mon[39523]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-10T06:44:01.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1847857192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc0ffe20 msgr2=0x7f0dbc1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:01.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1847857192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc1002a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009990 tx=0x7f0db0009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:01.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1847857192 shutdown_connections 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1847857192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dbc1007e0 0x7f0dbc10a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1847857192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc1033a0 0x7f0dbc1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1847857192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc1002a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1847857192 >> 172.21.15.184:0/1847857192 conn(0x7f0dbc0fb9c0 msgr2=0x7f0dbc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1847857192 shutdown_connections 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1847857192 wait complete. 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.445+0000 7f0dc1e9c700 1 Processor -- start 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 -- start start 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 0x7f0dbc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dbc1033a0 0x7f0dbc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19efe0 con 0x7f0dbc0ffe20 2024-02-10T06:44:01.448 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19f120 con 0x7f0dbc1007e0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dc1e9c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19f2b0 con 0x7f0dbc1033a0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 0x7f0dbc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 0x7f0dbc197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54504/0 (socket says 172.21.15.184:54504) 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 -- 172.21.15.184:0/1427547090 learned_addr learned my addr 172.21.15.184:0/1427547090 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbb7fe700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbbfff700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dbc1033a0 0x7f0dbc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 -- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dbc1033a0 msgr2=0x7f0dbc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dbc1033a0 0x7f0dbc19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 -- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc0ffe20 msgr2=0x7f0dbc197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 -- 172.21.15.184:0/1427547090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00095f0 con 0x7f0dbc1007e0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbb7fe700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.446+0000 7f0dbaffd700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 0x7f0dbc197990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0db0006010 tx=0x7f0db00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.447+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db001d070 con 0x7f0dbc1007e0 2024-02-10T06:44:01.449 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.447+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f0db0004110 con 0x7f0dbc1007e0 2024-02-10T06:44:01.450 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.447+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc100ff0 con 0x7f0dbc1007e0 2024-02-10T06:44:01.450 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.447+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0022d60 con 0x7f0dbc1007e0 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.447+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc101490 con 0x7f0dbc1007e0 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.448+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f0db0022500 con 0x7f0dbc1007e0 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.448+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc108b20 con 0x7f0dbc1007e0 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.449+0000 7f0db8ff9700 1 --2- 172.21.15.184:0/1427547090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f0da406d870 0x7f0da406fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.449+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4081+0+0 (secure 0 0 0) 0x7f0db008fbe0 con 0x7f0dbc1007e0 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.449+0000 7f0dbb7fe700 1 --2- 172.21.15.184:0/1427547090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f0da406d870 0x7f0da406fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:01.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.449+0000 7f0dbb7fe700 1 --2- 172.21.15.184:0/1427547090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f0da406d870 0x7f0da406fd30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005e10 tx=0x7f0dac005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:01.454 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.453+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0db0093060 con 0x7f0dbc1007e0 2024-02-10T06:44:01.644 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:01.643+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f0dbc10cef0 con 0x7f0dbc1007e0 2024-02-10T06:44:01.692 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 podman[67885]: 2024-02-10 06:44:01.265242641 +0000 UTC m=+0.018121528 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T06:44:01.692 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 podman[67885]: 2024-02-10 06:44:01.363374592 +0000 UTC m=+0.116253475 container create a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T06:44:01.692 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 podman[67885]: 2024-02-10 06:44:01.583194189 +0000 UTC m=+0.336073068 container init a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:44:01.692 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 podman[67885]: 2024-02-10 06:44:01.58883368 +0000 UTC m=+0.341712565 container start a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a) 2024-02-10T06:44:01.692 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 bash[67885]: a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 2024-02-10T06:44:01.988 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 06:44:01 smithi174 systemd[1]: Started Ceph rgw.foo.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:44:02.120 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.119+0000 7f0db8ff9700 1 -- 172.21.15.184:0/1427547090 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v59) v1 ==== 160+0+0 (secure 0 0 0) 0x7f0db0057bd0 con 0x7f0dbc1007e0 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f0da406d870 msgr2=0x7f0da406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1427547090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f0da406d870 0x7f0da406fd30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005e10 tx=0x7f0dac005d20 comp rx=0 tx=0).stop 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 msgr2=0x7f0dbc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 0x7f0dbc197990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0db0006010 tx=0x7f0db00057f0 comp rx=0 tx=0).stop 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 shutdown_connections 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dbc1033a0 0x7f0dbc19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:02.125 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dbc0ffe20 0x7f0dbc197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:02.126 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1427547090 >> v2:172.21.15.174:6800/2183136494 conn(0x7f0da406d870 0x7f0da406fd30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:02.126 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 --2- 172.21.15.184:0/1427547090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dbc1007e0 0x7f0dbc197990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:02.126 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 >> 172.21.15.184:0/1427547090 conn(0x7f0dbc0fb9c0 msgr2=0x7f0dbc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:02.126 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 shutdown_connections 2024-02-10T06:44:02.126 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:02.124+0000 7f0dc1e9c700 1 -- 172.21.15.184:0/1427547090 wait complete. 2024-02-10T06:44:02.126 INFO:teuthology.orchestra.run.smithi184.stderr:pool 'datapool' created 2024-02-10T06:44:02.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/1427547090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T06:44:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/1427547090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T06:44:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/1427547090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T06:44:02.783 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- rbd pool init datapool 2024-02-10T06:44:03.020 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: osdmap e59: 8 total, 8 up, 8 in 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3617581538' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: Saving service rgw.foo.a spec with placement smithi174=foo.a;count:1 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:03.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: pgmap v189: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-10T06:44:03.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:03 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: osdmap e59: 8 total, 8 up, 8 in 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3617581538' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: Saving service rgw.foo.a spec with placement smithi174=foo.a;count:1 2024-02-10T06:44:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: pgmap v189: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: osdmap e59: 8 total, 8 up, 8 in 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3617581538' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: Saving service rgw.foo.a spec with placement smithi174=foo.a;count:1 2024-02-10T06:44:03.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:03.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:03.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: pgmap v189: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-02-10T06:44:03.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:03 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: osdmap e60: 8 total, 8 up, 8 in 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:04 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: osdmap e60: 8 total, 8 up, 8 in 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: osdmap e60: 8 total, 8 up, 8 in 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:04 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: Checking dashboard <-> RGW credentials 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: osdmap e61: 8 total, 8 up, 8 in 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: pgmap v192: 68 pgs: 5 creating+peering, 6 active+clean, 57 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/166773098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T06:44:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: Checking dashboard <-> RGW credentials 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: osdmap e61: 8 total, 8 up, 8 in 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: pgmap v192: 68 pgs: 5 creating+peering, 6 active+clean, 57 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/166773098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T06:44:05.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: Checking dashboard <-> RGW credentials 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: osdmap e61: 8 total, 8 up, 8 in 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: pgmap v192: 68 pgs: 5 creating+peering, 6 active+clean, 57 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/166773098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T06:44:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T06:44:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:44:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:44:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-10T06:44:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[29878]: osdmap e62: 8 total, 8 up, 8 in 2024-02-10T06:44:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:44:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:44:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-10T06:44:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:06 smithi174 ceph-mon[39523]: osdmap e62: 8 total, 8 up, 8 in 2024-02-10T06:44:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:44:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:06 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T06:44:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-10T06:44:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:06 smithi184 ceph-mon[33513]: osdmap e62: 8 total, 8 up, 8 in 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: osdmap e63: 8 total, 8 up, 8 in 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: pgmap v195: 100 pgs: 10 creating+peering, 37 active+clean, 53 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:44:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[29878]: osdmap e64: 8 total, 8 up, 8 in 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: osdmap e63: 8 total, 8 up, 8 in 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: pgmap v195: 100 pgs: 10 creating+peering, 37 active+clean, 53 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:44:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:07 smithi174 ceph-mon[39523]: osdmap e64: 8 total, 8 up, 8 in 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: osdmap e63: 8 total, 8 up, 8 in 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: pgmap v195: 100 pgs: 10 creating+peering, 37 active+clean, 53 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T06:44:07.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:07 smithi184 ceph-mon[33513]: osdmap e64: 8 total, 8 up, 8 in 2024-02-10T06:44:07.885 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.184 --placement '1;smithi184=iscsi.a' 2024-02-10T06:44:08.130 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:09.276 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:09 smithi184 ceph-mon[33513]: osdmap e65: 8 total, 8 up, 8 in 2024-02-10T06:44:09.277 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.277 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.277 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:09 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.277 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.277 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:09 smithi184 ceph-mon[33513]: pgmap v198: 132 pgs: 14 creating+peering, 65 active+clean, 53 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T06:44:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[29878]: osdmap e65: 8 total, 8 up, 8 in 2024-02-10T06:44:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[29878]: pgmap v198: 132 pgs: 14 creating+peering, 65 active+clean, 53 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[39523]: osdmap e65: 8 total, 8 up, 8 in 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T06:44:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:09 smithi174 ceph-mon[39523]: pgmap v198: 132 pgs: 14 creating+peering, 65 active+clean, 53 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T06:44:09.792 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 -- 172.21.15.184:0/2975364674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 msgr2=0x7fbef8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 --2- 172.21.15.184:0/2975364674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8107580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009990 tx=0x7fbee8009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 -- 172.21.15.184:0/2975364674 shutdown_connections 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 --2- 172.21.15.184:0/2975364674 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbef8107ac0 0x7fbef8109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 --2- 172.21.15.184:0/2975364674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef8102860 0x7fbef8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 --2- 172.21.15.184:0/2975364674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8107580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 -- 172.21.15.184:0/2975364674 >> 172.21.15.184:0/2975364674 conn(0x7fbef80fc210 msgr2=0x7fbef80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 -- 172.21.15.184:0/2975364674 shutdown_connections 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.790+0000 7fbf007fc700 1 -- 172.21.15.184:0/2975364674 wait complete. 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 Processor -- start 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 -- start start 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbef8102860 0x7fbef8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:09.793 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef8107ac0 0x7fbef819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef819f000 con 0x7fbef8107ac0 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef819f140 con 0x7fbef8105190 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbf007fc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef819f2d0 con 0x7fbef8102860 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:54544/0 (socket says 172.21.15.184:54544) 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 -- 172.21.15.184:0/1838788774 learned_addr learned my addr 172.21.15.184:0/1838788774 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 -- 172.21.15.184:0/1838788774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbef8102860 msgr2=0x7fbef8197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbef8102860 0x7fbef8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 -- 172.21.15.184:0/1838788774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef8107ac0 msgr2=0x7fbef819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef8107ac0 0x7fbef819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.791+0000 7fbefdd97700 1 -- 172.21.15.184:0/1838788774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80095f0 con 0x7fbef8105190 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.792+0000 7fbefdd97700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8197a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbee80036c0 tx=0x7fbee8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.792+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee801d070 con 0x7fbef8105190 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.792+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fbee8022470 con 0x7fbef8105190 2024-02-10T06:44:09.794 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.792+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef819f550 con 0x7fbef8105190 2024-02-10T06:44:09.795 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.792+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee800f6c0 con 0x7fbef8105190 2024-02-10T06:44:09.795 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.792+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef819f9e0 con 0x7fbef8105190 2024-02-10T06:44:09.796 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.793+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef8050230 con 0x7fbef8105190 2024-02-10T06:44:09.796 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.793+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7fbee8003df0 con 0x7fbef8105190 2024-02-10T06:44:09.796 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.794+0000 7fbeef7fe700 1 --2- 172.21.15.184:0/1838788774 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbee406da30 0x7fbee406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:09.796 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.794+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5926+0+0 (secure 0 0 0) 0x7fbee808e620 con 0x7fbef8105190 2024-02-10T06:44:09.796 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.794+0000 7fbefe598700 1 --2- 172.21.15.184:0/1838788774 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbee406da30 0x7fbee406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:09.796 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.795+0000 7fbefe598700 1 --2- 172.21.15.184:0/1838788774 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbee406da30 0x7fbee406fef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbef40097b0 tx=0x7fbef4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:09.799 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:09.798+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbee8058110 con 0x7fbef8105190 2024-02-10T06:44:10.006 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.006+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef8109690 con 0x7fbee406da30 2024-02-10T06:44:10.015 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.014+0000 7fbeef7fe700 1 -- 172.21.15.184:0/1838788774 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7fbef8109690 con 0x7fbee406da30 2024-02-10T06:44:10.015 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled iscsi.datapool update... 2024-02-10T06:44:10.017 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbee406da30 msgr2=0x7fbee406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 --2- 172.21.15.184:0/1838788774 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbee406da30 0x7fbee406fef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbef40097b0 tx=0x7fbef4006d20 comp rx=0 tx=0).stop 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 msgr2=0x7fbef8197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8197a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbee80036c0 tx=0x7fbee8004730 comp rx=0 tx=0).stop 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 shutdown_connections 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef8107ac0 0x7fbef819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbef8102860 0x7fbef8197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 --2- 172.21.15.184:0/1838788774 >> v2:172.21.15.174:6800/2183136494 conn(0x7fbee406da30 0x7fbee406fef0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 --2- 172.21.15.184:0/1838788774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef8105190 0x7fbef8197a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 >> 172.21.15.184:0/1838788774 conn(0x7fbef80fc210 msgr2=0x7fbef8107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 shutdown_connections 2024-02-10T06:44:10.018 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:10.017+0000 7fbf007fc700 1 -- 172.21.15.184:0/1838788774 wait complete. 2024-02-10T06:44:10.333 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: osdmap e66: 8 total, 8 up, 8 in 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.334 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:10 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: osdmap e66: 8 total, 8 up, 8 in 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:44:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T06:44:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: osdmap e66: 8 total, 8 up, 8 in 2024-02-10T06:44:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1071078232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2995390865' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T06:44:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:10 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:10.680 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-10T06:44:10.680 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:44:10.680 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-10T06:44:10.710 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:44:10.710 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-10T06:44:10.739 DEBUG:teuthology.orchestra.run.smithi184:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@iscsi.iscsi.a.service 2024-02-10T06:44:10.782 INFO:tasks.cephadm:Adding prometheus.a on smithi184 2024-02-10T06:44:10.782 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply prometheus '1;smithi184=a' 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: osdmap e67: 8 total, 8 up, 8 in 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: pgmap v201: 132 pgs: 9 creating+peering, 84 active+clean, 39 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.330 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T06:44:11.331 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T06:44:11.331 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:11.331 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:11 smithi184 ceph-mon[33513]: Deploying daemon iscsi.iscsi.a on smithi184 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: osdmap e67: 8 total, 8 up, 8 in 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: pgmap v201: 132 pgs: 9 creating+peering, 84 active+clean, 39 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:44:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[29878]: Deploying daemon iscsi.iscsi.a on smithi184 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.184", "placement": "1;smithi184=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: Saving service iscsi.datapool spec with placement smithi184=iscsi.a;count:1 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: osdmap e67: 8 total, 8 up, 8 in 2024-02-10T06:44:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: pgmap v201: 132 pgs: 9 creating+peering, 84 active+clean, 39 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:11.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:11 smithi174 ceph-mon[39523]: Deploying daemon iscsi.iscsi.a on smithi184 2024-02-10T06:44:12.046 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:12.292 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:12 smithi184 ceph-mon[33513]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:44:12.293 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:12 smithi184 ceph-mon[33513]: Cluster is now healthy 2024-02-10T06:44:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:12 smithi174 ceph-mon[29878]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:44:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:12 smithi174 ceph-mon[29878]: Cluster is now healthy 2024-02-10T06:44:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:12 smithi174 ceph-mon[39523]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T06:44:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:12 smithi174 ceph-mon[39523]: Cluster is now healthy 2024-02-10T06:44:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:13 smithi184 ceph-mon[33513]: pgmap v202: 132 pgs: 128 active+clean, 4 unknown; 581 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 5.4 KiB/s wr, 127 op/s 2024-02-10T06:44:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:13 smithi174 ceph-mon[29878]: pgmap v202: 132 pgs: 128 active+clean, 4 unknown; 581 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 5.4 KiB/s wr, 127 op/s 2024-02-10T06:44:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:13 smithi174 ceph-mon[39523]: pgmap v202: 132 pgs: 128 active+clean, 4 unknown; 581 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 5.4 KiB/s wr, 127 op/s 2024-02-10T06:44:14.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 podman[62192]: 2024-02-10 06:44:14.161841865 +0000 UTC m=+0.687650862 container start 4090fe60e7f66764eea836f50d241eb7b2d5e42b671aeaf79be9abc217c79c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=main-290212a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T06:44:14.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 bash[62192]: 4090fe60e7f66764eea836f50d241eb7b2d5e42b671aeaf79be9abc217c79c2a 2024-02-10T06:44:14.498 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 -- 172.21.15.184:0/1504973623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3230105100 msgr2=0x7f32301074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 --2- 172.21.15.184:0/1504973623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3230105100 0x7f32301074f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f322c009990 tx=0x7f322c009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 -- 172.21.15.184:0/1504973623 shutdown_connections 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 --2- 172.21.15.184:0/1504973623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230107a30 0x7f3230109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 --2- 172.21.15.184:0/1504973623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32301027d0 0x7f3230104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 --2- 172.21.15.184:0/1504973623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3230105100 0x7f32301074f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.496+0000 7f3237e6d700 1 -- 172.21.15.184:0/1504973623 >> 172.21.15.184:0/1504973623 conn(0x7f32300fc1e0 msgr2=0x7f32300fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.497+0000 7f3237e6d700 1 -- 172.21.15.184:0/1504973623 shutdown_connections 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.497+0000 7f3237e6d700 1 -- 172.21.15.184:0/1504973623 wait complete. 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.497+0000 7f3237e6d700 1 Processor -- start 2024-02-10T06:44:14.499 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.497+0000 7f3237e6d700 1 -- start start 2024-02-10T06:44:14.500 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 systemd[1]: Started Ceph iscsi.iscsi.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:44:14.500 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug Started the configuration object watcher 2024-02-10T06:44:14.500 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug Checking for config object changes every 1s 2024-02-10T06:44:14.500 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug Processing osd blocklist entries for this node 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3237e6d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 0x7f32301973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3237e6d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230105100 0x7f3230197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3237e6d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3230107a30 0x7f323019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3237e6d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323019ef60 con 0x7f3230105100 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3237e6d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323019f0a0 con 0x7f32301027d0 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 0x7f32301973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235408700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230105100 0x7f3230197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:14.503 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 0x7f32301973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59092/0 (socket says 172.21.15.184:59092) 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 -- 172.21.15.184:0/75431455 learned_addr learned my addr 172.21.15.184:0/75431455 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f323640a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3230107a30 0x7f323019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323019f230 con 0x7f3230107a30 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 -- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3230107a30 msgr2=0x7f323019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3230107a30 0x7f323019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 -- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230105100 msgr2=0x7f3230197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230105100 0x7f3230197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235c09700 1 -- 172.21.15.184:0/75431455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0095f0 con 0x7f32301027d0 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.498+0000 7f3235408700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230105100 0x7f3230197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.499+0000 7f3235c09700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 0x7f32301973d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f322000c920 tx=0x7f322000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:14.504 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.499+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220004a30 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.499+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323019f510 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.499+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f32200054d0 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.499+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220007bc0 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.499+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323019fa60 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.500+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32300689e0 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.501+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 91372+0+0 (secure 0 0 0) 0x7f3220007d20 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.501+0000 7f3226ffd700 1 --2- 172.21.15.184:0/75431455 >> v2:172.21.15.174:6800/2183136494 conn(0x7f321c06d990 0x7f321c06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.501+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f322008cbf0 con 0x7f32301027d0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.501+0000 7f3235408700 1 --2- 172.21.15.184:0/75431455 >> v2:172.21.15.174:6800/2183136494 conn(0x7f321c06d990 0x7f321c06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:14.505 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.502+0000 7f3235408700 1 --2- 172.21.15.184:0/75431455 >> v2:172.21.15.174:6800/2183136494 conn(0x7f321c06d990 0x7f321c06fe50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b5c0 tx=0x7f322c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:14.507 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.507+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32200567f0 con 0x7f32301027d0 2024-02-10T06:44:14.741 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.740+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f32301095b0 con 0x7f321c06d990 2024-02-10T06:44:14.749 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.748+0000 7f3226ffd700 1 -- 172.21.15.184:0/75431455 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f32301095b0 con 0x7f321c06d990 2024-02-10T06:44:14.749 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled prometheus update... 2024-02-10T06:44:14.751 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 >> v2:172.21.15.174:6800/2183136494 conn(0x7f321c06d990 msgr2=0x7f321c06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:14.751 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 --2- 172.21.15.184:0/75431455 >> v2:172.21.15.174:6800/2183136494 conn(0x7f321c06d990 0x7f321c06fe50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b5c0 tx=0x7f322c005fb0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 msgr2=0x7f32301973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 0x7f32301973d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f322000c920 tx=0x7f322000cc30 comp rx=0 tx=0).stop 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 shutdown_connections 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3230107a30 0x7f323019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 --2- 172.21.15.184:0/75431455 >> v2:172.21.15.174:6800/2183136494 conn(0x7f321c06d990 0x7f321c06fe50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32301027d0 0x7f32301973d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 --2- 172.21.15.184:0/75431455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3230105100 0x7f3230197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.750+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 >> 172.21.15.184:0/75431455 conn(0x7f32300fc1e0 msgr2=0x7f3230107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.751+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 shutdown_connections 2024-02-10T06:44:14.752 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:14.751+0000 7f3237e6d700 1 -- 172.21.15.184:0/75431455 wait complete. 2024-02-10T06:44:15.111 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug Reading the configuration object to update local LIO configuration 2024-02-10T06:44:15.112 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug Configuration does not have an entry for this host(smithi184.front.sepia.ceph.com) - nothing to define to LIO 2024-02-10T06:44:15.112 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:14 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: pgmap v203: 132 pgs: 132 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 5.2 KiB/s wr, 136 op/s 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/3423206947' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T06:44:15.439 DEBUG:teuthology.orchestra.run.smithi184:prometheus.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@prometheus.a.service 2024-02-10T06:44:15.441 INFO:tasks.cephadm:Adding node-exporter.a on smithi174 2024-02-10T06:44:15.441 INFO:tasks.cephadm:Adding node-exporter.b on smithi184 2024-02-10T06:44:15.441 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply node-exporter '2;smithi174=a;smithi184=b' 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: pgmap v203: 132 pgs: 132 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 5.2 KiB/s wr, 136 op/s 2024-02-10T06:44:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/3423206947' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: pgmap v203: 132 pgs: 132 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 5.2 KiB/s wr, 136 op/s 2024-02-10T06:44:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:15.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:15.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/3423206947' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T06:44:15.781 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:16.462 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:16 smithi184 ceph-mon[33513]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:16.463 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:16 smithi184 ceph-mon[33513]: Saving service prometheus spec with placement smithi184=a;count:1 2024-02-10T06:44:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:16 smithi174 ceph-mon[29878]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:16 smithi174 ceph-mon[29878]: Saving service prometheus spec with placement smithi184=a;count:1 2024-02-10T06:44:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:16 smithi174 ceph-mon[39523]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:16 smithi174 ceph-mon[39523]: Saving service prometheus spec with placement smithi184=a;count:1 2024-02-10T06:44:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:17 smithi184 ceph-mon[33513]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 142 op/s 2024-02-10T06:44:17.720 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:17 smithi184 ceph-mon[33513]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-10T06:44:17.727 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 -- 172.21.15.184:0/2046294126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8105900 msgr2=0x7f01c8105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:17.727 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 --2- 172.21.15.184:0/2046294126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8105900 0x7f01c8105d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01c000b0a0 tx=0x7f01c000b3b0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.727 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 -- 172.21.15.184:0/2046294126 shutdown_connections 2024-02-10T06:44:17.727 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 --2- 172.21.15.184:0/2046294126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8105900 0x7f01c8105d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.727 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 --2- 172.21.15.184:0/2046294126 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c81005c0 0x7f01c81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.727 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 --2- 172.21.15.184:0/2046294126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8102f30 0x7f01c81053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 -- 172.21.15.184:0/2046294126 >> 172.21.15.184:0/2046294126 conn(0x7f01c80fc1e0 msgr2=0x7f01c80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 -- 172.21.15.184:0/2046294126 shutdown_connections 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.724+0000 7f01ce60a700 1 -- 172.21.15.184:0/2046294126 wait complete. 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 Processor -- start 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 -- start start 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c81005c0 0x7f01c819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 0x7f01c819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c8105900 0x7f01c81a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c81a5800 con 0x7f01c81005c0 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c81a5940 con 0x7f01c8102f30 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01ce60a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c81a5b00 con 0x7f01c8105900 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.725+0000 7f01c77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 0x7f01c819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:17.728 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 0x7f01c819e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59136/0 (socket says 172.21.15.184:59136) 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 -- 172.21.15.184:0/1529864468 learned_addr learned my addr 172.21.15.184:0/1529864468 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c7fff700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c81005c0 0x7f01c819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01ccba7700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c8105900 0x7f01c81a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 -- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c8105900 msgr2=0x7f01c81a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c8105900 0x7f01c81a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 -- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c81005c0 msgr2=0x7f01c819dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c81005c0 0x7f01c819dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 -- 172.21.15.184:0/1529864468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c0009d00 con 0x7f01c8102f30 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c7fff700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c81005c0 0x7f01c819dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c77fe700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 0x7f01c819e1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b530 tx=0x7f01bc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc011830 con 0x7f01c8102f30 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c81a5de0 con 0x7f01c8102f30 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f01bc011e40 con 0x7f01c8102f30 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc010ea0 con 0x7f01c8102f30 2024-02-10T06:44:17.729 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.726+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c81a62d0 con 0x7f01c8102f30 2024-02-10T06:44:17.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.727+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c80689e0 con 0x7f01c8102f30 2024-02-10T06:44:17.730 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.728+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7f01bc011990 con 0x7f01c8102f30 2024-02-10T06:44:17.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.728+0000 7f01c57fa700 1 --2- 172.21.15.184:0/1529864468 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01b006da20 0x7f01b006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:17.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.728+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f01bc08ccf0 con 0x7f01c8102f30 2024-02-10T06:44:17.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.728+0000 7f01c7fff700 1 --2- 172.21.15.184:0/1529864468 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01b006da20 0x7f01b006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:17.731 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.729+0000 7f01c7fff700 1 --2- 172.21.15.184:0/1529864468 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01b006da20 0x7f01b006fee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f01c819eb60 tx=0x7f01b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:17.733 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.733+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01bc056730 con 0x7f01c8102f30 2024-02-10T06:44:17.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:17 smithi174 ceph-mon[29878]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 142 op/s 2024-02-10T06:44:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:17 smithi174 ceph-mon[29878]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-10T06:44:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:17 smithi174 ceph-mon[39523]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 142 op/s 2024-02-10T06:44:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:17 smithi174 ceph-mon[39523]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-10T06:44:17.948 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.947+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi174=a;smithi184=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f01c8104ad0 con 0x7f01b006da20 2024-02-10T06:44:17.957 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.957+0000 7f01c57fa700 1 -- 172.21.15.184:0/1529864468 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f01c8104ad0 con 0x7f01b006da20 2024-02-10T06:44:17.958 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled node-exporter update... 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01b006da20 msgr2=0x7f01b006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 --2- 172.21.15.184:0/1529864468 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01b006da20 0x7f01b006fee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f01c819eb60 tx=0x7f01b8008040 comp rx=0 tx=0).stop 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 msgr2=0x7f01c819e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 0x7f01c819e1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b530 tx=0x7f01bc00b8f0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 shutdown_connections 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c8105900 0x7f01c81a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c81005c0 0x7f01c819dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 --2- 172.21.15.184:0/1529864468 >> v2:172.21.15.174:6800/2183136494 conn(0x7f01b006da20 0x7f01b006fee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.960 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 --2- 172.21.15.184:0/1529864468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c8102f30 0x7f01c819e1c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:17.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 >> 172.21.15.184:0/1529864468 conn(0x7f01c80fc1e0 msgr2=0x7f01c81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:17.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 shutdown_connections 2024-02-10T06:44:17.961 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:17.959+0000 7f01ce60a700 1 -- 172.21.15.184:0/1529864468 wait complete. 2024-02-10T06:44:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:18 smithi184 ceph-mon[33513]: osdmap e68: 8 total, 8 up, 8 in 2024-02-10T06:44:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:18 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:18 smithi174 ceph-mon[29878]: osdmap e68: 8 total, 8 up, 8 in 2024-02-10T06:44:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:18 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:18 smithi174 ceph-mon[39523]: osdmap e68: 8 total, 8 up, 8 in 2024-02-10T06:44:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:18 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:18.878 DEBUG:teuthology.orchestra.run.smithi174:node-exporter.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.a.service 2024-02-10T06:44:18.881 DEBUG:teuthology.orchestra.run.smithi184:node-exporter.b> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.b.service 2024-02-10T06:44:18.883 INFO:tasks.cephadm:Adding alertmanager.a on smithi174 2024-02-10T06:44:18.883 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply alertmanager '1;smithi174=a' 2024-02-10T06:44:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:19 smithi184 ceph-mon[33513]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi174=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:19 smithi184 ceph-mon[33513]: Saving service node-exporter spec with placement smithi174=a;smithi184=b;count:2 2024-02-10T06:44:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:19 smithi184 ceph-mon[33513]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 138 op/s 2024-02-10T06:44:19.735 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:19.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:19 smithi174 ceph-mon[29878]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi174=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:19 smithi174 ceph-mon[29878]: Saving service node-exporter spec with placement smithi174=a;smithi184=b;count:2 2024-02-10T06:44:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:19 smithi174 ceph-mon[29878]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 138 op/s 2024-02-10T06:44:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:19 smithi174 ceph-mon[39523]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi174=a;smithi184=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:19 smithi174 ceph-mon[39523]: Saving service node-exporter spec with placement smithi174=a;smithi184=b;count:2 2024-02-10T06:44:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:19 smithi174 ceph-mon[39523]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 138 op/s 2024-02-10T06:44:20.802 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.802 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.802 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:20.803 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:20.803 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.803 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: Deploying daemon prometheus.a on smithi184 2024-02-10T06:44:20.803 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:20 smithi184 ceph-mon[33513]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 113 op/s 2024-02-10T06:44:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: Deploying daemon prometheus.a on smithi184 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[29878]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 113 op/s 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: Deploying daemon prometheus.a on smithi184 2024-02-10T06:44:20.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:20 smithi174 ceph-mon[39523]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.7 KiB/s wr, 113 op/s 2024-02-10T06:44:21.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 -- 172.21.15.184:0/2475750307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564102f50 msgr2=0x7f55641033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:21.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 --2- 172.21.15.184:0/2475750307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564102f50 0x7f55641033b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55580099c0 tx=0x7f5558009cd0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 -- 172.21.15.184:0/2475750307 shutdown_connections 2024-02-10T06:44:21.392 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 --2- 172.21.15.184:0/2475750307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 0x7f556410a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 --2- 172.21.15.184:0/2475750307 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5564101d50 0x7f5564102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 --2- 172.21.15.184:0/2475750307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564102f50 0x7f55641033b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 -- 172.21.15.184:0/2475750307 >> 172.21.15.184:0/2475750307 conn(0x7f55640fd330 msgr2=0x7f55640ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 -- 172.21.15.184:0/2475750307 shutdown_connections 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.368+0000 7f556a099700 1 -- 172.21.15.184:0/2475750307 wait complete. 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 Processor -- start 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 -- start start 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 0x7f5564197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5564102f50 0x7f5564197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 0x7f556419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556419efd0 con 0x7f55641038f0 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556419f110 con 0x7f5564101d50 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f556a099700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556419f2a0 con 0x7f5564102f50 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 0x7f5564197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 0x7f5564197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59148/0 (socket says 172.21.15.184:59148) 2024-02-10T06:44:21.393 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 -- 172.21.15.184:0/3901361281 learned_addr learned my addr 172.21.15.184:0/3901361281 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f5562ffd700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5564102f50 0x7f5564197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f5563fff700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 0x7f556419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 -- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5564102f50 msgr2=0x7f5564197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5564102f50 0x7f5564197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 -- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 msgr2=0x7f556419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 0x7f556419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f55637fe700 1 -- 172.21.15.184:0/3901361281 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55580095f0 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.369+0000 7f5563fff700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 0x7f556419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.370+0000 7f55637fe700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 0x7f5564197440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f555400e970 tx=0x7f555400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.370+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554009790 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.370+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556419f580 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.370+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f555400eec0 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.370+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554003c90 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.370+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556419fad0 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.371+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5564108930 con 0x7f5564101d50 2024-02-10T06:44:21.394 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.371+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7f5554003e30 con 0x7f5564101d50 2024-02-10T06:44:21.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.372+0000 7f5560ff9700 1 --2- 172.21.15.184:0/3901361281 >> v2:172.21.15.174:6800/2183136494 conn(0x7f554c06da20 0x7f554c06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:21.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.372+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5942+0+0 (secure 0 0 0) 0x7f5554014070 con 0x7f5564101d50 2024-02-10T06:44:21.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.372+0000 7f5562ffd700 1 --2- 172.21.15.184:0/3901361281 >> v2:172.21.15.174:6800/2183136494 conn(0x7f554c06da20 0x7f554c06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:21.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.372+0000 7f5562ffd700 1 --2- 172.21.15.184:0/3901361281 >> v2:172.21.15.174:6800/2183136494 conn(0x7f554c06da20 0x7f554c06fee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5558006010 tx=0x7f5558019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:21.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.376+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5554056e80 con 0x7f5564101d50 2024-02-10T06:44:21.582 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.581+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi174=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f556410d000 con 0x7f554c06da20 2024-02-10T06:44:21.589 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.588+0000 7f5560ff9700 1 -- 172.21.15.184:0/3901361281 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f556410d000 con 0x7f554c06da20 2024-02-10T06:44:21.589 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled alertmanager update... 2024-02-10T06:44:21.591 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 >> v2:172.21.15.174:6800/2183136494 conn(0x7f554c06da20 msgr2=0x7f554c06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:21.591 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 --2- 172.21.15.184:0/3901361281 >> v2:172.21.15.174:6800/2183136494 conn(0x7f554c06da20 0x7f554c06fee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5558006010 tx=0x7f5558019040 comp rx=0 tx=0).stop 2024-02-10T06:44:21.591 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 msgr2=0x7f5564197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:21.591 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 0x7f5564197440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f555400e970 tx=0x7f555400ec80 comp rx=0 tx=0).stop 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 shutdown_connections 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55641038f0 0x7f556419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 --2- 172.21.15.184:0/3901361281 >> v2:172.21.15.174:6800/2183136494 conn(0x7f554c06da20 0x7f554c06fee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5564101d50 0x7f5564197440 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 --2- 172.21.15.184:0/3901361281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5564102f50 0x7f5564197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 >> 172.21.15.184:0/3901361281 conn(0x7f55640fd330 msgr2=0x7f55640fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 shutdown_connections 2024-02-10T06:44:21.592 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:21.591+0000 7f556a099700 1 -- 172.21.15.184:0/3901361281 wait complete. 2024-02-10T06:44:22.497 DEBUG:teuthology.orchestra.run.smithi174:alertmanager.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@alertmanager.a.service 2024-02-10T06:44:22.500 INFO:tasks.cephadm:Adding grafana.a on smithi184 2024-02-10T06:44:22.500 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph orch apply grafana '1;smithi184=a' 2024-02-10T06:44:22.742 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi174=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: Saving service alertmanager spec with placement smithi174=a;count:1 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1023 B/s wr, 48 op/s 2024-02-10T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:22 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi174=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: Saving service alertmanager spec with placement smithi174=a;count:1 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-10T06:44:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1023 B/s wr, 48 op/s 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi174=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: Saving service alertmanager spec with placement smithi174=a;count:1 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]: dispatch 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1023 B/s wr, 48 op/s 2024-02-10T06:44:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:22 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:23.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2024-02-10T06:44:23.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-10T06:44:23.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-02-10T06:44:23.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:23 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-10T06:44:23.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:23 smithi184 ceph-mon[33513]: osdmap e69: 8 total, 8 up, 8 in 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[29878]: osdmap e69: 8 total, 8 up, 8 in 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 5]}]': finished 2024-02-10T06:44:23.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-10T06:44:23.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:23 smithi174 ceph-mon[39523]: osdmap e69: 8 total, 8 up, 8 in 2024-02-10T06:44:24.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:24 smithi184 ceph-mon[33513]: osdmap e70: 8 total, 8 up, 8 in 2024-02-10T06:44:24.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:24 smithi184 ceph-mon[33513]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-02-10T06:44:24.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:24 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:24.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:44:24.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:24 smithi174 ceph-mon[29878]: osdmap e70: 8 total, 8 up, 8 in 2024-02-10T06:44:24.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:24 smithi174 ceph-mon[29878]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-02-10T06:44:24.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:24 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:24.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:24 smithi174 ceph-mon[39523]: osdmap e70: 8 total, 8 up, 8 in 2024-02-10T06:44:24.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:24 smithi174 ceph-mon[39523]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-02-10T06:44:24.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:24 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:25.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/103106596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 msgr2=0x7f7d5810a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:25.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/103106596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d5810a1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000b0d0 tx=0x7f7d5000b3e0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/103106596 shutdown_connections 2024-02-10T06:44:25.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/103106596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d5810a1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.241 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/103106596 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 0x7f7d58102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/103106596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 0x7f7d581033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/103106596 >> 172.21.15.184:0/103106596 conn(0x7f7d580fd330 msgr2=0x7f7d580ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/103106596 shutdown_connections 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/103106596 wait complete. 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.178+0000 7f7d5ccf7700 1 Processor -- start 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 -- start start 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 0x7f7d58075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 0x7f7d58074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d58074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58074dd0 con 0x7f7d58102f50 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58074f40 con 0x7f7d581038f0 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d581a1570 con 0x7f7d58101d50 2024-02-10T06:44:25.242 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d58074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d58074650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:41116/0 (socket says 172.21.15.184:41116) 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 -- 172.21.15.184:0/683724565 learned_addr learned my addr 172.21.15.184:0/683724565 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d55d9b700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 0x7f7d58074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5659c700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 0x7f7d58075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 -- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 msgr2=0x7f7d58075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 0x7f7d58075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 -- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 msgr2=0x7f7d58074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 0x7f7d58074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 -- 172.21.15.184:0/683724565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50009d00 con 0x7f7d581038f0 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d55d9b700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 0x7f7d58074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5659c700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 0x7f7d58075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d56d9d700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d58074650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009850 tx=0x7f7d50009180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000e040 con 0x7f7d581038f0 2024-02-10T06:44:25.243 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d581a17f0 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f7d50009490 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5001b7b0 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.179+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581a1c00 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.180+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58108930 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.181+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7f7d5001b910 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.181+0000 7f7d477fe700 1 --2- 172.21.15.184:0/683724565 >> v2:172.21.15.174:6800/2183136494 conn(0x7f7d4006da20 0x7f7d4006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.181+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f7d5008e7c0 con 0x7f7d581038f0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.181+0000 7f7d5659c700 1 --2- 172.21.15.184:0/683724565 >> v2:172.21.15.174:6800/2183136494 conn(0x7f7d4006da20 0x7f7d4006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.182+0000 7f7d5659c700 1 --2- 172.21.15.184:0/683724565 >> v2:172.21.15.174:6800/2183136494 conn(0x7f7d4006da20 0x7f7d4006fee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005950 tx=0x7f7d480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:25.244 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.186+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d500580a0 con 0x7f7d581038f0 2024-02-10T06:44:25.401 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.400+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 --> v2:172.21.15.174:6800/2183136494 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d58062160 con 0x7f7d4006da20 2024-02-10T06:44:25.409 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.408+0000 7f7d477fe700 1 -- 172.21.15.184:0/683724565 <== mgr.14152 v2:172.21.15.174:6800/2183136494 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f7d58062160 con 0x7f7d4006da20 2024-02-10T06:44:25.409 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled grafana update... 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.410+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 >> v2:172.21.15.174:6800/2183136494 conn(0x7f7d4006da20 msgr2=0x7f7d4006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.410+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/683724565 >> v2:172.21.15.174:6800/2183136494 conn(0x7f7d4006da20 0x7f7d4006fee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005950 tx=0x7f7d480058e0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 msgr2=0x7f7d58074650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d58074650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009850 tx=0x7f7d50009180 comp rx=0 tx=0).stop 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 shutdown_connections 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/683724565 >> v2:172.21.15.174:6800/2183136494 conn(0x7f7d4006da20 0x7f7d4006fee0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d581038f0 0x7f7d58074650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d58101d50 0x7f7d58075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 --2- 172.21.15.184:0/683724565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58102f50 0x7f7d58074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 >> 172.21.15.184:0/683724565 conn(0x7f7d580fd330 msgr2=0x7f7d580fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 shutdown_connections 2024-02-10T06:44:25.412 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:25.411+0000 7f7d5ccf7700 1 -- 172.21.15.184:0/683724565 wait complete. 2024-02-10T06:44:26.253 DEBUG:teuthology.orchestra.run.smithi184:grafana.a> sudo journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@grafana.a.service 2024-02-10T06:44:26.256 INFO:tasks.cephadm:Setting up client nodes... 2024-02-10T06:44:26.256 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T06:44:26.505 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:26 smithi184 ceph-mon[33513]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:26 smithi184 ceph-mon[33513]: Saving service grafana spec with placement smithi184=a;count:1 2024-02-10T06:44:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:26 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:26.626 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:26 smithi174 ceph-mon[29878]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:26.627 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:26 smithi174 ceph-mon[29878]: Saving service grafana spec with placement smithi184=a;count:1 2024-02-10T06:44:26.627 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:26 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:26.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:26 smithi174 ceph-mon[39523]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi184=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T06:44:26.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:26 smithi174 ceph-mon[39523]: Saving service grafana spec with placement smithi184=a;count:1 2024-02-10T06:44:26.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:26 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:27.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:27 smithi184 ceph-mon[33513]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-02-10T06:44:27.727 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:27 smithi174 ceph-mon[29878]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-02-10T06:44:27.727 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:27 smithi174 ceph-mon[39523]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-02-10T06:44:28.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 -- 172.21.15.174:0/3743100730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101620 msgr2=0x7f9984101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:28.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 --2- 172.21.15.174:0/3743100730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101620 0x7f9984101a80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f99780098f0 tx=0x7f9978009c00 comp rx=0 tx=0).stop 2024-02-10T06:44:28.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 -- 172.21.15.174:0/3743100730 shutdown_connections 2024-02-10T06:44:28.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 --2- 172.21.15.174:0/3743100730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101fc0 0x7f9984108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 --2- 172.21.15.174:0/3743100730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9984100420 0x7f9984100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 --2- 172.21.15.174:0/3743100730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101620 0x7f9984101a80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 -- 172.21.15.174:0/3743100730 >> 172.21.15.174:0/3743100730 conn(0x7f99840fb9c0 msgr2=0x7f99840fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 -- 172.21.15.174:0/3743100730 shutdown_connections 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.242+0000 7f998924f700 1 -- 172.21.15.174:0/3743100730 wait complete. 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.243+0000 7f998924f700 1 Processor -- start 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.243+0000 7f998924f700 1 -- start start 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.243+0000 7f998924f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9984100420 0x7f9984197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.243+0000 7f998924f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 0x7f99841978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.243+0000 7f998924f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 0x7f998419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998924f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998419ef20 con 0x7f9984101fc0 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998924f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998419f060 con 0x7f9984100420 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998924f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998419f1f0 con 0x7f9984101620 2024-02-10T06:44:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 0x7f99841978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998359e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 0x7f998419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 0x7f99841978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:56198/0 (socket says 172.21.15.174:56198) 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f9982d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9984100420 0x7f9984197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 -- 172.21.15.174:0/2266178290 learned_addr learned my addr 172.21.15.174:0/2266178290 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998359e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 0x7f998419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43812/0 (socket says 172.21.15.174:43812) 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 -- 172.21.15.174:0/2266178290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9984100420 msgr2=0x7f9984197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9984100420 0x7f9984197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 -- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 msgr2=0x7f998419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 0x7f998419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 -- 172.21.15.174:0/2266178290 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009d00 con 0x7f9984101620 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998359e700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 0x7f998419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998259c700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 0x7f99841978d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9978009230 tx=0x7f997800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997801c070 con 0x7f9984101620 2024-02-10T06:44:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f99780054f0 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9978009520 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978004c90 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.244+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998419f6f0 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.245+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9984107710 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.246+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7f9978005040 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.246+0000 7f996bfff700 1 --2- 172.21.15.174:0/2266178290 >> v2:172.21.15.174:6800/2183136494 conn(0x7f996c06dac0 0x7f996c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.246+0000 7f9982d9d700 1 --2- 172.21.15.174:0/2266178290 >> v2:172.21.15.174:6800/2183136494 conn(0x7f996c06dac0 0x7f996c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.246+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f997808e7c0 con 0x7f9984101620 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.247+0000 7f9982d9d700 1 --2- 172.21.15.174:0/2266178290 >> v2:172.21.15.174:6800/2183136494 conn(0x7f996c06dac0 0x7f996c06ff80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9984101480 tx=0x7f9974007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.251+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99780580a0 con 0x7f9984101620 2024-02-10T06:44:28.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.522+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f99841070f0 con 0x7f9984101620 2024-02-10T06:44:28.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.526+0000 7f996bfff700 1 -- 172.21.15.174:0/2266178290 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f997805b760 con 0x7f9984101620 2024-02-10T06:44:28.528 INFO:teuthology.orchestra.run.smithi174.stdout:[client.0] 2024-02-10T06:44:28.528 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBMG8dl0AxJHxAAKmaBYR7NAncm4awupXJcSg== 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 >> v2:172.21.15.174:6800/2183136494 conn(0x7f996c06dac0 msgr2=0x7f996c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 --2- 172.21.15.174:0/2266178290 >> v2:172.21.15.174:6800/2183136494 conn(0x7f996c06dac0 0x7f996c06ff80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9984101480 tx=0x7f9974007400 comp rx=0 tx=0).stop 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 msgr2=0x7f99841978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 0x7f99841978d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9978009230 tx=0x7f997800b560 comp rx=0 tx=0).stop 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 shutdown_connections 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9984101fc0 0x7f998419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 --2- 172.21.15.174:0/2266178290 >> v2:172.21.15.174:6800/2183136494 conn(0x7f996c06dac0 0x7f996c06ff80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9984100420 0x7f9984197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 --2- 172.21.15.174:0/2266178290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9984101620 0x7f99841978d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 >> 172.21.15.174:0/2266178290 conn(0x7f99840fb9c0 msgr2=0x7f9984104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 shutdown_connections 2024-02-10T06:44:28.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:28.528+0000 7f998924f700 1 -- 172.21.15.174:0/2266178290 wait complete. 2024-02-10T06:44:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:28 smithi184 ceph-mon[33513]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-10T06:44:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2266178290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[29878]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2266178290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[39523]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2266178290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:44:29.355 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:44:29.355 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-10T06:44:29.355 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-10T06:44:29.402 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T06:44:30.307 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T06:44:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:31 smithi184 ceph-mon[33513]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 1 objects/s recovering 2024-02-10T06:44:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:31 smithi174 ceph-mon[29878]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 1 objects/s recovering 2024-02-10T06:44:31.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:31 smithi174 ceph-mon[39523]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 1 objects/s recovering 2024-02-10T06:44:33.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:33 smithi184 ceph-mon[33513]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-02-10T06:44:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:33 smithi174 ceph-mon[29878]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-02-10T06:44:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:33 smithi174 ceph-mon[39523]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 -- 172.21.15.184:0/368522990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 msgr2=0x7ff030104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 --2- 172.21.15.184:0/368522990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030104bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff024009990 tx=0x7ff024009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 -- 172.21.15.184:0/368522990 shutdown_connections 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 --2- 172.21.15.184:0/368522990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff030107a60 0x7ff030109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 --2- 172.21.15.184:0/368522990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030104bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 --2- 172.21.15.184:0/368522990 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff030105130 0x7ff030107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 -- 172.21.15.184:0/368522990 >> 172.21.15.184:0/368522990 conn(0x7ff0300fc210 msgr2=0x7ff0300fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 -- 172.21.15.184:0/368522990 shutdown_connections 2024-02-10T06:44:34.521 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.517+0000 7ff0374fb700 1 -- 172.21.15.184:0/368522990 wait complete. 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 Processor -- start 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 -- start start 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff030105130 0x7ff030197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff030107a60 0x7ff03019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03019eea0 con 0x7ff030107a60 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03019efe0 con 0x7ff030102800 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff0374fb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03019f170 con 0x7ff030105130 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff035297700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff035297700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:42562/0 (socket says 172.21.15.184:42562) 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff035297700 1 -- 172.21.15.184:0/132274425 learned_addr learned my addr 172.21.15.184:0/132274425 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff034a96700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff030105130 0x7ff030197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.518+0000 7ff035a98700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff030107a60 0x7ff03019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:34.522 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff035297700 1 -- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff030105130 msgr2=0x7ff030197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff035297700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff030105130 0x7ff030197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff035297700 1 -- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff030107a60 msgr2=0x7ff03019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff035297700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff030107a60 0x7ff03019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff035297700 1 -- 172.21.15.184:0/132274425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0240095f0 con 0x7ff030102800 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff035297700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030197310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff024006010 tx=0x7ff024004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02401d070 con 0x7ff030102800 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03019f3f0 con 0x7ff030102800 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff024004c90 con 0x7ff030102800 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024003b20 con 0x7ff030102800 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.519+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03019f810 con 0x7ff030102800 2024-02-10T06:44:34.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.520+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0300fdd90 con 0x7ff030102800 2024-02-10T06:44:34.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.521+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 91423+0+0 (secure 0 0 0) 0x7ff024005000 con 0x7ff030102800 2024-02-10T06:44:34.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.521+0000 7ff0227fc700 1 --2- 172.21.15.184:0/132274425 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff01c06dac0 0x7ff01c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:34.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.521+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6030+0+0 (secure 0 0 0) 0x7ff02408e210 con 0x7ff030102800 2024-02-10T06:44:34.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.522+0000 7ff034a96700 1 --2- 172.21.15.184:0/132274425 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff01c06dac0 0x7ff01c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:34.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.522+0000 7ff034a96700 1 --2- 172.21.15.184:0/132274425 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff01c06dac0 0x7ff01c06ff80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff02c005950 tx=0x7ff02c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:34.527 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.526+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff024057c10 con 0x7ff030102800 2024-02-10T06:44:34.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:44:34.810 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.809+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff03010cf00 con 0x7ff030102800 2024-02-10T06:44:34.815 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.814+0000 7ff0227fc700 1 -- 172.21.15.184:0/132274425 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff02405b2d0 con 0x7ff030102800 2024-02-10T06:44:34.815 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2024-02-10T06:44:34.815 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQBSG8dl33NiMBAAoQfqFiP33r7x2qIwISEOcA== 2024-02-10T06:44:34.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff01c06dac0 msgr2=0x7ff01c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:34.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 --2- 172.21.15.184:0/132274425 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff01c06dac0 0x7ff01c06ff80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff02c005950 tx=0x7ff02c00b410 comp rx=0 tx=0).stop 2024-02-10T06:44:34.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 msgr2=0x7ff030197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:34.817 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030197310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff024006010 tx=0x7ff024004730 comp rx=0 tx=0).stop 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 shutdown_connections 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff030107a60 0x7ff03019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 --2- 172.21.15.184:0/132274425 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff01c06dac0 0x7ff01c06ff80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff030102800 0x7ff030197310 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 --2- 172.21.15.184:0/132274425 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff030105130 0x7ff030197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 >> 172.21.15.184:0/132274425 conn(0x7ff0300fc210 msgr2=0x7ff030103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 shutdown_connections 2024-02-10T06:44:34.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-02-10T06:44:34.816+0000 7ff0374fb700 1 -- 172.21.15.184:0/132274425 wait complete. 2024-02-10T06:44:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[29878]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 948 B/s rd, 0 op/s; 30 B/s, 1 objects/s recovering 2024-02-10T06:44:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.184:0/132274425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:44:35.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[39523]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 948 B/s rd, 0 op/s; 30 B/s, 1 objects/s recovering 2024-02-10T06:44:35.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:35.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.184:0/132274425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:35.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:35.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:35 smithi184 ceph-mon[33513]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 948 B/s rd, 0 op/s; 30 B/s, 1 objects/s recovering 2024-02-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:35 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.184:0/132274425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T06:44:35.753 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:35 smithi184 systemd[1]: Starting Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:44:36.070 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-02-10T06:44:36.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-10T06:44:36.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-10T06:44:36.114 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-10T06:44:36.114 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-10T06:44:36.114 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph mgr dump --format=json 2024-02-10T06:44:36.363 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:36.581 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 podman[64272]: 2024-02-10 06:44:36.210774905 +0000 UTC m=+0.018719550 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-10T06:44:36.581 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 podman[64272]: 2024-02-10 06:44:36.320677571 +0000 UTC m=+0.128622201 container create aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:44:36.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 podman[64272]: 2024-02-10 06:44:36.581432174 +0000 UTC m=+0.389376805 container init aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:44:36.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 podman[64272]: 2024-02-10 06:44:36.587388282 +0000 UTC m=+0.395332913 container start aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:44:36.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.611Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-10T06:44:36.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.611Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.611Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi184 (none))" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.611Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.611Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.613Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.613Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.614Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.614Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.616Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.616Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.961µs 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.616Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.616Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.616Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.833µs wal_replay_duration=173.626µs wbl_replay_duration=90ns total_replay_duration=209.698µs 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.617Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.617Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-10T06:44:36.874 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.617Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T06:44:36.875 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.635Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.183709ms db_storage=755ns remote_storage=1.08µs web_handler=413ns query_engine=610ns scrape=387.825µs scrape_sd=75.613µs notify=750ns notify_sd=1.238µs rules=17.375179ms tracing=5.608µs 2024-02-10T06:44:36.875 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.635Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-10T06:44:36.875 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:44:36.635Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-10T06:44:36.875 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 bash[64272]: aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 2024-02-10T06:44:36.875 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:44:36 smithi184 systemd[1]: Started Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:44:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[29878]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T06:44:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[39523]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T06:44:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.769 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mon[33513]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T06:44:37.769 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.770 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.770 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.770 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T06:44:37.770 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' 2024-02-10T06:44:37.934 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ignoring --setuser ceph since I am not root 2024-02-10T06:44:37.934 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ignoring --setgroup ceph since I am not root 2024-02-10T06:44:38.123 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: ignoring --setuser ceph since I am not root 2024-02-10T06:44:38.123 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: ignoring --setgroup ceph since I am not root 2024-02-10T06:44:38.123 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:37 smithi184 ceph-mgr[35626]: -- 172.21.15.184:0/3494359089 <== mon.0 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579a2baa1a0 con 0x5579a2b80800 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 -- 172.21.15.174:0/2230134462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 msgr2=0x7efd0c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 --2- 172.21.15.174:0/2230134462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c101a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efd00009990 tx=0x7efd00009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 -- 172.21.15.174:0/2230134462 shutdown_connections 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 --2- 172.21.15.174:0/2230134462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 0x7efd0c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 --2- 172.21.15.174:0/2230134462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c101a60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 --2- 172.21.15.174:0/2230134462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 0x7efd0c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 -- 172.21.15.174:0/2230134462 >> 172.21.15.174:0/2230134462 conn(0x7efd0c077220 msgr2=0x7efd0c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 -- 172.21.15.174:0/2230134462 shutdown_connections 2024-02-10T06:44:38.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.200+0000 7efd11ee5700 1 -- 172.21.15.174:0/2230134462 wait complete. 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 Processor -- start 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 -- start start 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 0x7efd0c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 0x7efd0c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c197ec0 con 0x7efd0c105fa0 2024-02-10T06:44:38.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c198030 con 0x7efd0c102030 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd11ee5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c1981f0 con 0x7efd0c101680 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:43532/0 (socket says 172.21.15.174:43532) 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd0b7fe700 1 -- 172.21.15.174:0/1391663072 learned_addr learned my addr 172.21.15.174:0/1391663072 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd0affd700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 0x7efd0c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.201+0000 7efd0bfff700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 0x7efd0c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0b7fe700 1 -- 172.21.15.174:0/1391663072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 msgr2=0x7efd0c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0b7fe700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 0x7efd0c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0b7fe700 1 -- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 msgr2=0x7efd0c197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0b7fe700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 0x7efd0c197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0b7fe700 1 -- 172.21.15.174:0/1391663072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04009c40 con 0x7efd0c101680 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0bfff700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 0x7efd0c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0affd700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 0x7efd0c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:38.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd0b7fe700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c19d690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efd00000c00 tx=0x7efd00004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:38.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0001d070 con 0x7efd0c101680 2024-02-10T06:44:38.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7efd00022470 con 0x7efd0c101680 2024-02-10T06:44:38.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd11ee5700 1 -- 172.21.15.174:0/1391663072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd000095f0 con 0x7efd0c101680 2024-02-10T06:44:38.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00017650 con 0x7efd0c101680 2024-02-10T06:44:38.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.202+0000 7efd11ee5700 1 -- 172.21.15.174:0/1391663072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c1a44f0 con 0x7efd0c101680 2024-02-10T06:44:38.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.203+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec005120 con 0x7efd0c101680 2024-02-10T06:44:38.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.204+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 17) v1 ==== 91437+0+0 (secure 0 0 0) 0x7efd0000fa90 con 0x7efd0c101680 2024-02-10T06:44:38.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.204+0000 7efd08ff9700 1 --2- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 0x7efcf406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:38.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.204+0000 7efd0affd700 1 -- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 msgr2=0x7efcf406ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2183136494 2024-02-10T06:44:38.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.204+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6030+0+0 (secure 0 0 0) 0x7efd0008e640 con 0x7efd0c101680 2024-02-10T06:44:38.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.204+0000 7efd0affd700 1 --2- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 0x7efcf406ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:44:38.210 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.209+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd00058040 con 0x7efd0c101680 2024-02-10T06:44:38.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.405+0000 7efd0affd700 1 -- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 msgr2=0x7efcf406ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2183136494 2024-02-10T06:44:38.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.405+0000 7efd0affd700 1 --2- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 0x7efcf406ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:44:38.469 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.467+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efcec0056a0 con 0x7efd0c101680 2024-02-10T06:44:38.469 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:38.338+0000 7f8688d94200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:44:38.469 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:38.434+0000 7f8688d94200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:44:38.473 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.471+0000 7efd08ff9700 1 -- 172.21.15.174:0/1391663072 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+174817 (secure 0 0 0) 0x7efd0005b700 con 0x7efd0c101680 2024-02-10T06:44:38.475 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 msgr2=0x7efcf406ff30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 --2- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 0x7efcf406ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 msgr2=0x7efd0c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c19d690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efd00000c00 tx=0x7efd00004730 comp rx=0 tx=0).stop 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 shutdown_connections 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd0c105fa0 0x7efd0c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd0c101680 0x7efd0c19d690 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 --2- 172.21.15.174:0/1391663072 >> v2:172.21.15.174:6800/2183136494 conn(0x7efcf406da70 0x7efcf406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 --2- 172.21.15.174:0/1391663072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd0c102030 0x7efd0c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:38.477 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 >> 172.21.15.174:0/1391663072 conn(0x7efd0c077220 msgr2=0x7efd0c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:38.478 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 shutdown_connections 2024-02-10T06:44:38.478 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:38.475+0000 7efcf27fc700 1 -- 172.21.15.174:0/1391663072 wait complete. 2024-02-10T06:44:38.622 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:38.297+0000 7f3531a94200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T06:44:38.623 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:38.390+0000 7f3531a94200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T06:44:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-mon[29878]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T06:44:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-mon[29878]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-10T06:44:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1391663072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T06:44:38.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-mon[39523]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T06:44:38.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-mon[39523]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-10T06:44:38.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1391663072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T06:44:39.026 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:38 smithi184 ceph-mon[33513]: from='mgr.14152 172.21.15.174:0/4101195236' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T06:44:39.026 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:38 smithi184 ceph-mon[33513]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-10T06:44:39.026 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1391663072' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T06:44:39.334 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2183136494}]},"active_addr":"172.21.15.174:6800/2183136494","active_change":"2024-02-10T06:39:22.324124+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.174:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":91484631}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":2983552204}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":3967713004}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":498364371}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":426142081}]}]} 2024-02-10T06:44:39.338 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-10T06:44:39.338 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-10T06:44:39.338 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T06:44:39.358 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.025+0000 7f3531a94200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:44:39.358 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.117+0000 7f3531a94200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:44:39.358 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.262+0000 7f3531a94200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:44:39.364 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.094+0000 7f8688d94200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T06:44:39.364 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.187+0000 7f8688d94200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T06:44:39.364 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.338+0000 7f8688d94200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T06:44:39.586 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:39.623 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.357+0000 7f3531a94200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:44:39.623 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.440+0000 7f3531a94200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:44:39.708 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.439+0000 7f8688d94200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T06:44:39.708 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.530+0000 7f8688d94200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T06:44:39.985 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.810+0000 7f8688d94200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:44:39.985 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:39 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:39.926+0000 7f8688d94200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:44:39.993 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.702+0000 7f3531a94200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T06:44:39.993 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.811+0000 7f3531a94200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T06:44:39.993 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.898+0000 7f3531a94200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:44:40.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:40 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:40.016+0000 7f8688d94200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T06:44:40.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:40 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:40.113+0000 7f8688d94200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:44:40.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:39.992+0000 7f3531a94200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T06:44:40.373 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:40.168+0000 7f3531a94200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:44:40.738 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:40 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:40.292+0000 7f8688d94200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T06:44:41.425 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:41.286+0000 7f8688d94200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 -- 172.21.15.174:0/2661582780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b8100da0 msgr2=0x7ff7b810d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 --2- 172.21.15.174:0/2661582780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b8100da0 0x7ff7b810d2b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b0a0 tx=0x7ff7b000b3b0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 -- 172.21.15.174:0/2661582780 shutdown_connections 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 --2- 172.21.15.174:0/2661582780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b8100da0 0x7ff7b810d2b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 --2- 172.21.15.174:0/2661582780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7b8106470 0x7ff7b8106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 --2- 172.21.15.174:0/2661582780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7b81003e0 0x7ff7b8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 -- 172.21.15.174:0/2661582780 >> 172.21.15.174:0/2661582780 conn(0x7ff7b8077220 msgr2=0x7ff7b80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 -- 172.21.15.174:0/2661582780 shutdown_connections 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.446+0000 7ff7bede1700 1 -- 172.21.15.174:0/2661582780 wait complete. 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 Processor -- start 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 -- start start 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 0x7ff7b819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:41.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7b8100da0 0x7ff7b819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7b8106470 0x7ff7b8195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8195f00 con 0x7ff7b81003e0 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8196070 con 0x7ff7b8100da0 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bede1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8196230 con 0x7ff7b8106470 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bcb7d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 0x7ff7b819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bcb7d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 0x7ff7b819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39062/0 (socket says 172.21.15.174:39062) 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bcb7d700 1 -- 172.21.15.174:0/1592176556 learned_addr learned my addr 172.21.15.174:0/1592176556 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7b7fff700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7b8100da0 0x7ff7b819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.447+0000 7ff7bd37e700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7b8106470 0x7ff7b8195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bcb7d700 1 -- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7b8106470 msgr2=0x7ff7b8195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bcb7d700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7b8106470 0x7ff7b8195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bcb7d700 1 -- 172.21.15.174:0/1592176556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7b8100da0 msgr2=0x7ff7b819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bcb7d700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7b8100da0 0x7ff7b819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bcb7d700 1 -- 172.21.15.174:0/1592176556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0009d00 con 0x7ff7b81003e0 2024-02-10T06:44:41.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bcb7d700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 0x7ff7b819b630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d980 tx=0x7ff7ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:41.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004220 con 0x7ff7b81003e0 2024-02-10T06:44:41.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b8196510 con 0x7ff7b81003e0 2024-02-10T06:44:41.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff7ac007640 con 0x7ff7b81003e0 2024-02-10T06:44:41.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac005630 con 0x7ff7b81003e0 2024-02-10T06:44:41.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.448+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b81a25b0 con 0x7ff7b81003e0 2024-02-10T06:44:41.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.449+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b8108a00 con 0x7ff7b81003e0 2024-02-10T06:44:41.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.450+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 91437+0+0 (secure 0 0 0) 0x7ff7ac00f690 con 0x7ff7b81003e0 2024-02-10T06:44:41.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.450+0000 7ff7b5ffb700 1 --2- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 0x7ff7a8072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:41.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.450+0000 7ff7b7fff700 1 -- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 msgr2=0x7ff7a8072140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2183136494 2024-02-10T06:44:41.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.450+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6030+0+0 (secure 0 0 0) 0x7ff7ac08d000 con 0x7ff7b81003e0 2024-02-10T06:44:41.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.450+0000 7ff7b7fff700 1 --2- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 0x7ff7a8072140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:44:41.456 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.455+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7ac0568e0 con 0x7ff7b81003e0 2024-02-10T06:44:41.595 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:41.127+0000 7f3531a94200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T06:44:41.595 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:41.261+0000 7f3531a94200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:44:41.652 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.651+0000 7ff7b7fff700 1 -- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 msgr2=0x7ff7a8072140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2183136494 2024-02-10T06:44:41.652 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.652+0000 7ff7b7fff700 1 --2- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 0x7ff7a8072140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:44:41.656 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.655+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7b80689e0 con 0x7ff7b81003e0 2024-02-10T06:44:41.657 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.657+0000 7ff7b5ffb700 1 -- 172.21.15.174:0/1592176556 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20768 (secure 0 0 0) 0x7ff7ac059fa0 con 0x7ff7b81003e0 2024-02-10T06:44:41.658 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:44:41.658 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":70,"fsid":"df845a14-c7de-11ee-95b8-87774f69a715","created":"2024-02-10T06:38:25.649613+0000","modified":"2024-02-10T06:44:23.591663+0000","last_up_change":"2024-02-10T06:43:49.428578+0000","last_in_change":"2024-02-10T06:43:31.395302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T06:41:40.659218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-10T06:44:01.867332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-02-10T06:44:02.105607+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T06:44:03.431431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T06:44:05.437184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T06:44:07.501066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b892cc18-df62-424f-8250-d301c822e8f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6801","nonce":4183537518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4183537518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4183537518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4183537518}]},"public_addr":"172.21.15.174:6801/4183537518","cluster_addr":"172.21.15.174:6802/4183537518","heartbeat_back_addr":"172.21.15.174:6804/4183537518","heartbeat_front_addr":"172.21.15.174:6803/4183537518","state":["exists","up"]},{"osd":1,"uuid":"aebb7103-2298-403b-8df6-e078cf073484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6805","nonce":640275884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":640275884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":640275884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":640275884}]},"public_addr":"172.21.15.174:6805/640275884","cluster_addr":"172.21.15.174:6806/640275884","heartbeat_back_addr":"172.21.15.174:6808/640275884","heartbeat_front_addr":"172.21.15.174:6807/640275884","state":["exists","up"]},{"osd":2,"uuid":"11f1e7b8-5292-4c92-8823-1e9387e69d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6809","nonce":2318790055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2318790055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2318790055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":2318790055}]},"public_addr":"172.21.15.174:6809/2318790055","cluster_addr":"172.21.15.174:6810/2318790055","heartbeat_back_addr":"172.21.15.174:6812/2318790055","heartbeat_front_addr":"172.21.15.174:6811/2318790055","state":["exists","up"]},{"osd":3,"uuid":"6db6715d-ad5c-42a4-b051-0d2fdd2fef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6813","nonce":2745280023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2745280023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2745280023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6815","nonce":2745280023}]},"public_addr":"172.21.15.174:6813/2745280023","cluster_addr":"172.21.15.174:6814/2745280023","heartbeat_back_addr":"172.21.15.174:6816/2745280023","heartbeat_front_addr":"172.21.15.174:6815/2745280023","state":["exists","up"]},{"osd":4,"uuid":"3df6af28-331e-466d-ba68-0c7db4e6a616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3815653813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":3815653813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":3815653813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3815653813}]},"public_addr":"172.21.15.184:6800/3815653813","cluster_addr":"172.21.15.184:6801/3815653813","heartbeat_back_addr":"172.21.15.184:6803/3815653813","heartbeat_front_addr":"172.21.15.184:6802/3815653813","state":["exists","up"]},{"osd":5,"uuid":"21af74d5-65b4-4cfe-b5be-68d3ba4516b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2462414067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2462414067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":2462414067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2462414067}]},"public_addr":"172.21.15.184:6804/2462414067","cluster_addr":"172.21.15.184:6805/2462414067","heartbeat_back_addr":"172.21.15.184:6807/2462414067","heartbeat_front_addr":"172.21.15.184:6806/2462414067","state":["exists","up"]},{"osd":6,"uuid":"ffc0976a-c58c-4798-9c02-a1d7b828b46e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2589400030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2589400030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":2589400030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2589400030}]},"public_addr":"172.21.15.184:6808/2589400030","cluster_addr":"172.21.15.184:6809/2589400030","heartbeat_back_addr":"172.21.15.184:6811/2589400030","heartbeat_front_addr":"172.21.15.184:6810/2589400030","state":["exists","up"]},{"osd":7,"uuid":"ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3506789608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":3506789608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":3506789608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3506789608}]},"public_addr":"172.21.15.184:6812/3506789608","cluster_addr":"172.21.15.184:6813/3506789608","heartbeat_back_addr":"172.21.15.184:6815/3506789608","heartbeat_front_addr":"172.21.15.184:6814/3506789608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:40:46.684618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:41:12.730284+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:41:38.522191+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:04.175541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:28.672368+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:54.571801+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:43:20.849260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:43:47.772410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":7,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:0/3154304474":"2024-02-11T06:39:22.324013+0000","172.21.15.174:0/3421212038":"2024-02-11T06:39:22.324013+0000","172.21.15.174:6800/2787410738":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/1784760840":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/3867738525":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/2793344305":"2024-02-11T06:39:22.324013+0000","172.21.15.174:0/4219955631":"2024-02-11T06:38:55.602697+0000","172.21.15.174:6800/1048754011":"2024-02-11T06:39:22.324013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T06:44:41.660 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 msgr2=0x7ff7a8072140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:44:41.660 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 --2- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 0x7ff7a8072140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.660 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 msgr2=0x7ff7b819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:41.660 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 0x7ff7b819b630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d980 tx=0x7ff7ac00dc90 comp rx=0 tx=0).stop 2024-02-10T06:44:41.660 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 shutdown_connections 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7b8106470 0x7ff7b8195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7b81003e0 0x7ff7b819b630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 --2- 172.21.15.174:0/1592176556 >> v2:172.21.15.174:6800/2183136494 conn(0x7ff7a806fc80 0x7ff7a8072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 --2- 172.21.15.174:0/1592176556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7b8100da0 0x7ff7b819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 >> 172.21.15.174:0/1592176556 conn(0x7ff7b8077220 msgr2=0x7ff7b81033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 shutdown_connections 2024-02-10T06:44:41.661 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:41.660+0000 7ff7bede1700 1 -- 172.21.15.174:0/1592176556 wait complete. 2024-02-10T06:44:41.709 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:41.425+0000 7f8688d94200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T06:44:41.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1592176556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:44:41.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:41.594+0000 7f3531a94200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:44:41.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:41.684+0000 7f3531a94200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:44:41.988 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1592176556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:44:41.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1592176556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:44:41.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:41.776+0000 7f8688d94200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T06:44:41.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:41 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:41.869+0000 7f8688d94200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T06:44:42.292 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:42 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:42.071+0000 7f8688d94200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:44:42.292 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:42 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:42.261+0000 7f8688d94200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:44:42.293 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-10T06:44:42.294 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T06:44:42.345 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:41.878+0000 7f3531a94200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:44:42.345 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:42.061+0000 7f3531a94200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T06:44:42.582 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:42.607 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:42.344+0000 7f3531a94200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:44:42.845 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:42 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:42.572+0000 7f8688d94200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T06:44:42.872 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:42.606+0000 7f3531a94200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:44:42.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:42.841+0000 7f3531a94200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:44:43.109 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:42 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:42.844+0000 7f8688d94200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T06:44:43.109 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:43 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:43.093+0000 7f8688d94200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T06:44:44.069 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:43 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:43.918+0000 7f8688d94200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:44:44.122 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:43.641+0000 7f3531a94200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T06:44:44.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 -- 172.21.15.174:0/2045536028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98100c20 msgr2=0x7fcc98101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:44.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 --2- 172.21.15.174:0/2045536028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98100c20 0x7fcc98101080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009990 tx=0x7fcc88009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 -- 172.21.15.174:0/2045536028 shutdown_connections 2024-02-10T06:44:44.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 --2- 172.21.15.174:0/2045536028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc981015c0 0x7fcc9810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 --2- 172.21.15.174:0/2045536028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc98106c60 0x7fcc98107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 --2- 172.21.15.174:0/2045536028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98100c20 0x7fcc98101080 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 -- 172.21.15.174:0/2045536028 >> 172.21.15.174:0/2045536028 conn(0x7fcc98077b50 msgr2=0x7fcc98077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 -- 172.21.15.174:0/2045536028 shutdown_connections 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 -- 172.21.15.174:0/2045536028 wait complete. 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.303+0000 7fcca053e700 1 Processor -- start 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 -- start start 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 0x7fcc98198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc981015c0 0x7fcc98198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98106c60 0x7fcc981a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981995f0 con 0x7fcc98100c20 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a3e20 con 0x7fcc981015c0 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcca053e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a3f60 con 0x7fcc98106c60 2024-02-10T06:44:44.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 0x7fcc98198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 0x7fcc98198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39092/0 (socket says 172.21.15.174:39092) 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 -- 172.21.15.174:0/1975715027 learned_addr learned my addr 172.21.15.174:0/1975715027 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9dad9700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc981015c0 0x7fcc98198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9eadb700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98106c60 0x7fcc981a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 -- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98106c60 msgr2=0x7fcc981a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98106c60 0x7fcc981a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 -- 172.21.15.174:0/1975715027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc981015c0 msgr2=0x7fcc98198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc981015c0 0x7fcc98198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 -- 172.21.15.174:0/1975715027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90009c40 con 0x7fcc98100c20 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9dad9700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc981015c0 0x7fcc98198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.304+0000 7fcc9e2da700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 0x7fcc98198860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e9d0 tx=0x7fcc9400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.305+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94009790 con 0x7fcc98100c20 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.305+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc880095f0 con 0x7fcc98100c20 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.305+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fcc9400ee60 con 0x7fcc98100c20 2024-02-10T06:44:44.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.305+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94018420 con 0x7fcc98100c20 2024-02-10T06:44:44.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.305+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc981a4580 con 0x7fcc98100c20 2024-02-10T06:44:44.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.306+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc98109180 con 0x7fcc98100c20 2024-02-10T06:44:44.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.306+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 91437+0+0 (secure 0 0 0) 0x7fcc94010430 con 0x7fcc98100c20 2024-02-10T06:44:44.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.307+0000 7fcc8f7fe700 1 --2- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 0x7fcc8406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:44.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.307+0000 7fcc9dad9700 1 -- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 msgr2=0x7fcc8406ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2183136494 2024-02-10T06:44:44.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.307+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6030+0+0 (secure 0 0 0) 0x7fcc94014070 con 0x7fcc98100c20 2024-02-10T06:44:44.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.307+0000 7fcc9dad9700 1 --2- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 0x7fcc8406ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-10T06:44:44.312 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.311+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc94052d20 con 0x7fcc98100c20 2024-02-10T06:44:44.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.508+0000 7fcc9dad9700 1 -- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 msgr2=0x7fcc8406ff30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.174:6800/2183136494 2024-02-10T06:44:44.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.508+0000 7fcc9dad9700 1 --2- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 0x7fcc8406ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-10T06:44:44.512 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.511+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc980689e0 con 0x7fcc98100c20 2024-02-10T06:44:44.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.513+0000 7fcc8f7fe700 1 -- 172.21.15.174:0/1975715027 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20768 (secure 0 0 0) 0x7fcc94056e40 con 0x7fcc98100c20 2024-02-10T06:44:44.514 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:44:44.514 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":70,"fsid":"df845a14-c7de-11ee-95b8-87774f69a715","created":"2024-02-10T06:38:25.649613+0000","modified":"2024-02-10T06:44:23.591663+0000","last_up_change":"2024-02-10T06:43:49.428578+0000","last_in_change":"2024-02-10T06:43:31.395302+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T06:41:40.659218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-10T06:44:01.867332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-02-10T06:44:02.105607+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T06:44:03.431431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T06:44:05.437184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T06:44:07.501066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b892cc18-df62-424f-8250-d301c822e8f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6801","nonce":4183537518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4183537518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4183537518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4183537518}]},"public_addr":"172.21.15.174:6801/4183537518","cluster_addr":"172.21.15.174:6802/4183537518","heartbeat_back_addr":"172.21.15.174:6804/4183537518","heartbeat_front_addr":"172.21.15.174:6803/4183537518","state":["exists","up"]},{"osd":1,"uuid":"aebb7103-2298-403b-8df6-e078cf073484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6805","nonce":640275884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":640275884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":640275884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":640275884}]},"public_addr":"172.21.15.174:6805/640275884","cluster_addr":"172.21.15.174:6806/640275884","heartbeat_back_addr":"172.21.15.174:6808/640275884","heartbeat_front_addr":"172.21.15.174:6807/640275884","state":["exists","up"]},{"osd":2,"uuid":"11f1e7b8-5292-4c92-8823-1e9387e69d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6809","nonce":2318790055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2318790055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2318790055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":2318790055}]},"public_addr":"172.21.15.174:6809/2318790055","cluster_addr":"172.21.15.174:6810/2318790055","heartbeat_back_addr":"172.21.15.174:6812/2318790055","heartbeat_front_addr":"172.21.15.174:6811/2318790055","state":["exists","up"]},{"osd":3,"uuid":"6db6715d-ad5c-42a4-b051-0d2fdd2fef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6813","nonce":2745280023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2745280023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2745280023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6815","nonce":2745280023}]},"public_addr":"172.21.15.174:6813/2745280023","cluster_addr":"172.21.15.174:6814/2745280023","heartbeat_back_addr":"172.21.15.174:6816/2745280023","heartbeat_front_addr":"172.21.15.174:6815/2745280023","state":["exists","up"]},{"osd":4,"uuid":"3df6af28-331e-466d-ba68-0c7db4e6a616","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3815653813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6801","nonce":3815653813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6803","nonce":3815653813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3815653813}]},"public_addr":"172.21.15.184:6800/3815653813","cluster_addr":"172.21.15.184:6801/3815653813","heartbeat_back_addr":"172.21.15.184:6803/3815653813","heartbeat_front_addr":"172.21.15.184:6802/3815653813","state":["exists","up"]},{"osd":5,"uuid":"21af74d5-65b4-4cfe-b5be-68d3ba4516b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2462414067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":2462414067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6807","nonce":2462414067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2462414067}]},"public_addr":"172.21.15.184:6804/2462414067","cluster_addr":"172.21.15.184:6805/2462414067","heartbeat_back_addr":"172.21.15.184:6807/2462414067","heartbeat_front_addr":"172.21.15.184:6806/2462414067","state":["exists","up"]},{"osd":6,"uuid":"ffc0976a-c58c-4798-9c02-a1d7b828b46e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":2589400030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":2589400030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":2589400030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":2589400030}]},"public_addr":"172.21.15.184:6808/2589400030","cluster_addr":"172.21.15.184:6809/2589400030","heartbeat_back_addr":"172.21.15.184:6811/2589400030","heartbeat_front_addr":"172.21.15.184:6810/2589400030","state":["exists","up"]},{"osd":7,"uuid":"ae0e80cd-d1e4-439d-ad57-c2f3078b3a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3506789608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6813","nonce":3506789608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":3506789608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3506789608}]},"public_addr":"172.21.15.184:6812/3506789608","cluster_addr":"172.21.15.184:6813/3506789608","heartbeat_back_addr":"172.21.15.184:6815/3506789608","heartbeat_front_addr":"172.21.15.184:6814/3506789608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:40:46.684618+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:41:12.730284+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:41:38.522191+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:04.175541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:28.672368+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:42:54.571801+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:43:20.849260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T06:43:47.772410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1d","mappings":[{"from":7,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:0/3154304474":"2024-02-11T06:39:22.324013+0000","172.21.15.174:0/3421212038":"2024-02-11T06:39:22.324013+0000","172.21.15.174:6800/2787410738":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/1784760840":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/3867738525":"2024-02-11T06:38:55.602697+0000","172.21.15.174:0/2793344305":"2024-02-11T06:39:22.324013+0000","172.21.15.174:0/4219955631":"2024-02-11T06:38:55.602697+0000","172.21.15.174:6800/1048754011":"2024-02-11T06:39:22.324013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T06:44:44.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 msgr2=0x7fcc8406ff30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T06:44:44.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 --2- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 0x7fcc8406ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 msgr2=0x7fcc98198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:44.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 0x7fcc98198860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e9d0 tx=0x7fcc9400ece0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 shutdown_connections 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc98106c60 0x7fcc981a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc98100c20 0x7fcc98198860 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 --2- 172.21.15.174:0/1975715027 >> v2:172.21.15.174:6800/2183136494 conn(0x7fcc8406da70 0x7fcc8406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 --2- 172.21.15.174:0/1975715027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc981015c0 0x7fcc98198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.515+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 >> 172.21.15.174:0/1975715027 conn(0x7fcc98077b50 msgr2=0x7fcc980ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.516+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 shutdown_connections 2024-02-10T06:44:44.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:44.516+0000 7fcca053e700 1 -- 172.21.15.174:0/1975715027 wait complete. 2024-02-10T06:44:44.533 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:44.214+0000 7f3531a94200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:44:44.533 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:44.314+0000 7f3531a94200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:44:44.850 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:44.516+0000 7f8688d94200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T06:44:44.850 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:44.626+0000 7f8688d94200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T06:44:44.850 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1975715027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:44:44.850 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: Standby manager daemon x restarted 2024-02-10T06:44:44.850 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: Standby manager daemon x started 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[39523]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1975715027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: Standby manager daemon x restarted 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: Standby manager daemon x started 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:44:44.851 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-mon[29878]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1975715027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: Standby manager daemon x restarted 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: Standby manager daemon x started 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-mon[33513]: from='mgr.? 172.21.15.184:0/2141748244' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:44.532+0000 7f3531a94200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: [10/Feb/2024:06:44:44] ENGINE Bus STARTING 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: CherryPy Checker: 2024-02-10T06:44:44.873 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: The Application mounted at '' has an empty config. 2024-02-10T06:44:44.874 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: 2024-02-10T06:44:44.874 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: [10/Feb/2024:06:44:44] ENGINE Serving on http://:::9283 2024-02-10T06:44:44.874 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 06:44:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x[35603]: [10/Feb/2024:06:44:44] ENGINE Bus STARTED 2024-02-10T06:44:45.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:44.849+0000 7f8688d94200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T06:44:45.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:44 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:44:44] ENGINE Bus STARTING 2024-02-10T06:44:45.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: CherryPy Checker: 2024-02-10T06:44:45.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: The Application mounted at '' has an empty config. 2024-02-10T06:44:45.240 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T06:44:45.240 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:44:45] ENGINE Serving on http://:::9283 2024-02-10T06:44:45.240 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:44:45] ENGINE Bus STARTED 2024-02-10T06:44:45.466 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-10T06:44:45.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-10T06:44:45.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-10T06:44:45.468 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-10T06:44:45.468 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-10T06:44:45.469 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-10T06:44:45.469 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-10T06:44:45.469 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-10T06:44:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-10T06:44:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: Active manager daemon y restarted 2024-02-10T06:44:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: Activating manager daemon y 2024-02-10T06:44:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: osdmap e71: 8 total, 8 up, 8 in 2024-02-10T06:44:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: mgrmap e19: y(active, starting, since 0.0113117s), standbys: x 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:44:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: Manager daemon y is now available 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:44:45.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: Active manager daemon y restarted 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: Activating manager daemon y 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: osdmap e71: 8 total, 8 up, 8 in 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: mgrmap e19: y(active, starting, since 0.0113117s), standbys: x 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:44:45.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:44:45.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: Manager daemon y is now available 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:44:45.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: Active manager daemon y restarted 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: Activating manager daemon y 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: osdmap e71: 8 total, 8 up, 8 in 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: mgrmap e19: y(active, starting, since 0.0113117s), standbys: x 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T06:44:45.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: Manager daemon y is now available 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:44:45.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T06:44:45.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T06:44:46.023 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.023 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.024 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.029 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.029 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.034 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.035 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.060 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:44:46.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[29878]: [10/Feb/2024:06:44:45] ENGINE Bus STARTING 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[29878]: [10/Feb/2024:06:44:45] ENGINE Serving on https://172.21.15.174:7150 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[29878]: [10/Feb/2024:06:44:45] ENGINE Serving on http://172.21.15.174:8765 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[29878]: [10/Feb/2024:06:44:45] ENGINE Bus STARTED 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[29878]: mgrmap e20: y(active, since 1.01755s), standbys: x 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[39523]: [10/Feb/2024:06:44:45] ENGINE Bus STARTING 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[39523]: [10/Feb/2024:06:44:45] ENGINE Serving on https://172.21.15.174:7150 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[39523]: [10/Feb/2024:06:44:45] ENGINE Serving on http://172.21.15.174:8765 2024-02-10T06:44:46.809 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[39523]: [10/Feb/2024:06:44:45] ENGINE Bus STARTED 2024-02-10T06:44:46.810 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:46 smithi174 ceph-mon[39523]: mgrmap e20: y(active, since 1.01755s), standbys: x 2024-02-10T06:44:46.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:46 smithi184 ceph-mon[33513]: [10/Feb/2024:06:44:45] ENGINE Bus STARTING 2024-02-10T06:44:46.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:46 smithi184 ceph-mon[33513]: [10/Feb/2024:06:44:45] ENGINE Serving on https://172.21.15.174:7150 2024-02-10T06:44:46.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:46 smithi184 ceph-mon[33513]: [10/Feb/2024:06:44:45] ENGINE Serving on http://172.21.15.174:8765 2024-02-10T06:44:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:46 smithi184 ceph-mon[33513]: [10/Feb/2024:06:44:45] ENGINE Bus STARTED 2024-02-10T06:44:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:46 smithi184 ceph-mon[33513]: mgrmap e20: y(active, since 1.01755s), standbys: x 2024-02-10T06:44:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:47 smithi174 ceph-mon[29878]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:47 smithi174 ceph-mon[39523]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:47.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:47 smithi184 ceph-mon[33513]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:48.178 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:44:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:44:48.734 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[29878]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[29878]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[39523]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[39523]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:48.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:48 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:48.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:48 smithi184 ceph-mon[33513]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:48 smithi184 ceph-mon[33513]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-10T06:44:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:48 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:48 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi184 to 3760M 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[29878]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi184 to 3760M 2024-02-10T06:44:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:50 smithi174 ceph-mon[39523]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi184 to 3760M 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:44:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:50 smithi184 ceph-mon[33513]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-10T06:44:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:52 smithi174 ceph-mon[29878]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:52 smithi174 ceph-mon[39523]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:52.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:52 smithi184 ceph-mon[33513]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:44:53.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:53 smithi174 ceph-mon[29878]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-10T06:44:53.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:53 smithi174 ceph-mon[39523]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-10T06:44:54.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:53 smithi184 ceph-mon[33513]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-10T06:44:54.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 -- 172.21.15.174:0/4201111879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006afd0 msgr2=0x7fd8a0106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:54.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/4201111879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006afd0 0x7fd8a0106960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009990 tx=0x7fd88c009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 -- 172.21.15.174:0/4201111879 shutdown_connections 2024-02-10T06:44:54.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/4201111879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8a0107090 0x7fd8a010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/4201111879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8a006a620 0x7fd8a006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/4201111879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006afd0 0x7fd8a0106960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 -- 172.21.15.174:0/4201111879 >> 172.21.15.174:0/4201111879 conn(0x7fd8a0077b90 msgr2=0x7fd8a0077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 -- 172.21.15.174:0/4201111879 shutdown_connections 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.906+0000 7fd8a6b84700 1 -- 172.21.15.174:0/4201111879 wait complete. 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 Processor -- start 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 -- start start 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006a620 0x7fd8a019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8a006afd0 0x7fd8a019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8a0107090 0x7fd8a01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0197e50 con 0x7fd8a006a620 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0197fc0 con 0x7fd8a0107090 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a6b84700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0198180 con 0x7fd8a006afd0 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a4920700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006a620 0x7fd8a019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8a006afd0 0x7fd8a019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:54.913 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a4920700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006a620 0x7fd8a019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38862/0 (socket says 172.21.15.174:38862) 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a4920700 1 -- 172.21.15.174:0/1379458627 learned_addr learned my addr 172.21.15.174:0/1379458627 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a5121700 1 --2- 172.21.15.174:0/1379458627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8a0107090 0x7fd8a01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 -- 172.21.15.174:0/1379458627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8a0107090 msgr2=0x7fd8a01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 --2- 172.21.15.174:0/1379458627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8a0107090 0x7fd8a01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 -- 172.21.15.174:0/1379458627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006a620 msgr2=0x7fd8a019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 --2- 172.21.15.174:0/1379458627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006a620 0x7fd8a019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 -- 172.21.15.174:0/1379458627 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898009c40 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd8a4920700 1 --2- 172.21.15.174:0/1379458627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a006a620 0x7fd8a019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.907+0000 7fd89ffff700 1 --2- 172.21.15.174:0/1379458627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8a006afd0 0x7fd8a019db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd88c000c00 tx=0x7fd88c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.908+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c01d070 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.908+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd88c00b680 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.908+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c022d80 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.908+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c0095f0 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.908+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a01a4480 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.909+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8a01117d0 con 0x7fd8a006afd0 2024-02-10T06:44:54.914 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.909+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fd88c022470 con 0x7fd8a006afd0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd89dffb700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd89006da80 0x7fd89006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fd88c08e850 con 0x7fd8a006afd0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd8a4920700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd89006da80 0x7fd89006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd89dffb700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.184:6808/2589400030 conn(0x7fd890077420 0x7fd8900798c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 --> v2:172.21.15.184:6808/2589400030 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd890079f50 con 0x7fd890077420 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88c0932a0 con 0x7fd8a006afd0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd8a5121700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.184:6808/2589400030 conn(0x7fd890077420 0x7fd8900798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd8a4920700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd89006da80 0x7fd89006ff40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd894005950 tx=0x7fd89400b500 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.910+0000 7fd8a5121700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.184:6808/2589400030 conn(0x7fd890077420 0x7fd8900798c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:54.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.911+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== osd.6 v2:172.21.15.184:6808/2589400030 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd890079f50 con 0x7fd890077420 2024-02-10T06:44:54.928 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.927+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 --> v2:172.21.15.184:6808/2589400030 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd8a00623b0 con 0x7fd890077420 2024-02-10T06:44:54.928 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd89dffb700 1 -- 172.21.15.174:0/1379458627 <== osd.6 v2:172.21.15.184:6808/2589400030 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd8a00623b0 con 0x7fd890077420 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 >> v2:172.21.15.184:6808/2589400030 conn(0x7fd890077420 msgr2=0x7fd8900798c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.184:6808/2589400030 conn(0x7fd890077420 0x7fd8900798c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd89006da80 msgr2=0x7fd89006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/1379458627 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd89006da80 0x7fd89006ff40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd894005950 tx=0x7fd89400b500 comp rx=0 tx=0).stop 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8a006afd0 msgr2=0x7fd8a019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 --2- 172.21.15.174:0/1379458627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8a006afd0 0x7fd8a019db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd88c000c00 tx=0x7fd88c005740 comp rx=0 tx=0).stop 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a5121700 1 -- 172.21.15.174:0/1379458627 reap_dead start 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 shutdown_connections 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 >> 172.21.15.174:0/1379458627 conn(0x7fd8a0077b90 msgr2=0x7fd8a00fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 shutdown_connections 2024-02-10T06:44:54.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:54.928+0000 7fd8a6b84700 1 -- 172.21.15.174:0/1379458627 wait complete. 2024-02-10T06:44:55.915 INFO:teuthology.orchestra.run.smithi174.stdout:197568495636 2024-02-10T06:44:55.916 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-10T06:44:56.152 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:55 smithi174 ceph-mon[29878]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-10T06:44:56.152 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:55 smithi174 ceph-mon[39523]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-10T06:44:56.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:55 smithi184 ceph-mon[33513]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-10T06:44:56.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:44:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:44:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 -- 172.21.15.174:0/480028106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c100310 msgr2=0x7f513c100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 --2- 172.21.15.174:0/480028106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c100310 0x7f513c100790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f512c009990 tx=0x7f512c009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 -- 172.21.15.174:0/480028106 shutdown_connections 2024-02-10T06:44:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 --2- 172.21.15.174:0/480028106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 0x7f513c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 --2- 172.21.15.174:0/480028106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f513c106d90 0x7f513c107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 --2- 172.21.15.174:0/480028106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c100310 0x7f513c100790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.635+0000 7f5143ca1700 1 -- 172.21.15.174:0/480028106 >> 172.21.15.174:0/480028106 conn(0x7f513c077220 msgr2=0x7f513c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 -- 172.21.15.174:0/480028106 shutdown_connections 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 -- 172.21.15.174:0/480028106 wait complete. 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 Processor -- start 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 -- start start 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f513c100310 0x7f513c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 0x7f513c101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.636+0000 7f5143ca1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c106d90 0x7f513c101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f5143ca1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1a5e50 con 0x7f513c106d90 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f5143ca1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1a5fc0 con 0x7f513c100310 2024-02-10T06:44:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f5143ca1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1a6180 con 0x7f513c100cd0 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514123c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 0x7f513c101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c106d90 0x7f513c101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514123c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 0x7f513c101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59420/0 (socket says 172.21.15.174:59420) 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514123c700 1 -- 172.21.15.174:0/4168687197 learned_addr learned my addr 172.21.15.174:0/4168687197 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f5141a3d700 1 --2- 172.21.15.174:0/4168687197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f513c100310 0x7f513c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 -- 172.21.15.174:0/4168687197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 msgr2=0x7f513c101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 --2- 172.21.15.174:0/4168687197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 0x7f513c101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 -- 172.21.15.174:0/4168687197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f513c100310 msgr2=0x7f513c1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 --2- 172.21.15.174:0/4168687197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f513c100310 0x7f513c1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 -- 172.21.15.174:0/4168687197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134009c40 con 0x7f513c106d90 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514123c700 1 --2- 172.21.15.174:0/4168687197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f513c100cd0 0x7f513c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f514223e700 1 --2- 172.21.15.174:0/4168687197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c106d90 0x7f513c101f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f513400ef70 tx=0x7f513400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.637+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134018070 con 0x7f513c106d90 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.638+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f513400f040 con 0x7f513c106d90 2024-02-10T06:44:56.640 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.638+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0095f0 con 0x7f513c106d90 2024-02-10T06:44:56.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.638+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134004280 con 0x7f513c106d90 2024-02-10T06:44:56.641 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.638+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c1a6720 con 0x7f513c106d90 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.639+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f513c111930 con 0x7f513c106d90 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.639+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f5134003a00 con 0x7f513c106d90 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5132ffd700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6800/1252702393 conn(0x7f512806da80 0x7f512806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f513408cd30 con 0x7f513c106d90 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5141a3d700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6800/1252702393 conn(0x7f512806da80 0x7f512806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5132ffd700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6813/2745280023 conn(0x7f51280773a0 0x7f5128079840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 --> v2:172.21.15.174:6813/2745280023 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5128079ed0 con 0x7f51280773a0 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f513408f3d0 con 0x7f513c106d90 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f514123c700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6813/2745280023 conn(0x7f51280773a0 0x7f5128079840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.640+0000 7f5141a3d700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6800/1252702393 conn(0x7f512806da80 0x7f512806ff40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5138009be0 tx=0x7f5138009260 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:56.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.641+0000 7f514123c700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6813/2745280023 conn(0x7f51280773a0 0x7f5128079840 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:56.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.641+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== osd.3 v2:172.21.15.174:6813/2745280023 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5128079ed0 con 0x7f51280773a0 2024-02-10T06:44:56.664 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.663+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 --> v2:172.21.15.174:6813/2745280023 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f513c0623b0 con 0x7f51280773a0 2024-02-10T06:44:56.664 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5132ffd700 1 -- 172.21.15.174:0/4168687197 <== osd.3 v2:172.21.15.174:6813/2745280023 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f513c0623b0 con 0x7f51280773a0 2024-02-10T06:44:56.665 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6813/2745280023 conn(0x7f51280773a0 msgr2=0x7f5128079840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.665 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5143ca1700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6813/2745280023 conn(0x7f51280773a0 0x7f5128079840 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.665 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6800/1252702393 conn(0x7f512806da80 msgr2=0x7f512806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5143ca1700 1 --2- 172.21.15.174:0/4168687197 >> v2:172.21.15.174:6800/1252702393 conn(0x7f512806da80 0x7f512806ff40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5138009be0 tx=0x7f5138009260 comp rx=0 tx=0).stop 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c106d90 msgr2=0x7f513c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.664+0000 7f5143ca1700 1 --2- 172.21.15.174:0/4168687197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f513c106d90 0x7f513c101f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f513400ef70 tx=0x7f513400c3a0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.665+0000 7f514223e700 1 -- 172.21.15.174:0/4168687197 reap_dead start 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.665+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 shutdown_connections 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.665+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 >> 172.21.15.174:0/4168687197 conn(0x7f513c077220 msgr2=0x7f513c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.665+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 shutdown_connections 2024-02-10T06:44:56.666 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.665+0000 7f5143ca1700 1 -- 172.21.15.174:0/4168687197 wait complete. 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 -- 172.21.15.174:0/1424849319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100cd0 msgr2=0x7f206c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 --2- 172.21.15.174:0/1424849319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100cd0 0x7f206c10ec30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f206400b0a0 tx=0x7f206400b3b0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 -- 172.21.15.174:0/1424849319 shutdown_connections 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 --2- 172.21.15.174:0/1424849319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100cd0 0x7f206c10ec30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 --2- 172.21.15.174:0/1424849319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f206c106d90 0x7f206c107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 --2- 172.21.15.174:0/1424849319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f206c100310 0x7f206c100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 -- 172.21.15.174:0/1424849319 >> 172.21.15.174:0/1424849319 conn(0x7f206c077220 msgr2=0x7f206c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 -- 172.21.15.174:0/1424849319 shutdown_connections 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.867+0000 7f20746f4700 1 -- 172.21.15.174:0/1424849319 wait complete. 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.868+0000 7f20746f4700 1 Processor -- start 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.868+0000 7f20746f4700 1 -- start start 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.868+0000 7f20746f4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100310 0x7f206c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f206c100cd0 0x7f206c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f206c106d90 0x7f206c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c197fe0 con 0x7f206c100310 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c198120 con 0x7f206c106d90 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1982e0 con 0x7f206c100cd0 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100310 0x7f206c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100310 0x7f206c19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41072/0 (socket says 172.21.15.174:41072) 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 -- 172.21.15.174:0/2638692070 learned_addr learned my addr 172.21.15.174:0/2638692070 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:56.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2071c8f700 1 --2- 172.21.15.174:0/2638692070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f206c100cd0 0x7f206c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 -- 172.21.15.174:0/2638692070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f206c100cd0 msgr2=0x7f206c19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 --2- 172.21.15.174:0/2638692070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f206c100cd0 0x7f206c19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072c91700 1 --2- 172.21.15.174:0/2638692070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f206c106d90 0x7f206c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 -- 172.21.15.174:0/2638692070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f206c106d90 msgr2=0x7f206c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 --2- 172.21.15.174:0/2638692070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f206c106d90 0x7f206c197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 -- 172.21.15.174:0/2638692070 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068009520 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f2072490700 1 --2- 172.21.15.174:0/2638692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100310 0x7f206c19d7c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d980 tx=0x7f205c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c004220 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f205c007640 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0055f0 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2064009d00 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.869+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1a4690 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.870+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f206c111930 con 0x7f206c100310 2024-02-10T06:44:56.874 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.871+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f205c004750 con 0x7f206c100310 2024-02-10T06:44:56.875 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f20637fe700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6800/1252702393 conn(0x7f205806da80 0x7f205806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f205c08d190 con 0x7f206c100310 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f2071c8f700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6800/1252702393 conn(0x7f205806da80 0x7f205806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f20637fe700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6805/640275884 conn(0x7f2058077420 0x7f20580798c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 --> v2:172.21.15.174:6805/640275884 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2058079f50 con 0x7f2058077420 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f205c08f830 con 0x7f206c100310 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f2072c91700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6805/640275884 conn(0x7f2058077420 0x7f20580798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.872+0000 7f2071c8f700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6800/1252702393 conn(0x7f205806da80 0x7f205806ff40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f206c0ff8f0 tx=0x7f2068009260 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.873+0000 7f2072c91700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6805/640275884 conn(0x7f2058077420 0x7f20580798c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:56.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.873+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== osd.1 v2:172.21.15.174:6805/640275884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2058079f50 con 0x7f2058077420 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.898+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 --> v2:172.21.15.174:6805/640275884 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f206c0623b0 con 0x7f2058077420 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.898+0000 7f20637fe700 1 -- 172.21.15.174:0/2638692070 <== osd.1 v2:172.21.15.174:6805/640275884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f206c0623b0 con 0x7f2058077420 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6805/640275884 conn(0x7f2058077420 msgr2=0x7f20580798c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6805/640275884 conn(0x7f2058077420 0x7f20580798c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6800/1252702393 conn(0x7f205806da80 msgr2=0x7f205806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 --2- 172.21.15.174:0/2638692070 >> v2:172.21.15.174:6800/1252702393 conn(0x7f205806da80 0x7f205806ff40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f206c0ff8f0 tx=0x7f2068009260 comp rx=0 tx=0).stop 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100310 msgr2=0x7f206c19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 --2- 172.21.15.174:0/2638692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100310 0x7f206c19d7c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d980 tx=0x7f205c00dc90 comp rx=0 tx=0).stop 2024-02-10T06:44:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f2072c91700 1 -- 172.21.15.174:0/2638692070 reap_dead start 2024-02-10T06:44:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 shutdown_connections 2024-02-10T06:44:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 >> 172.21.15.174:0/2638692070 conn(0x7f206c077220 msgr2=0x7f206c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 shutdown_connections 2024-02-10T06:44:56.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:56.899+0000 7f20746f4700 1 -- 172.21.15.174:0/2638692070 wait complete. 2024-02-10T06:44:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:56 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:56 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:56 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:44:57.914 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:44:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:44:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:44:57.941 INFO:teuthology.orchestra.run.smithi174.stdout:55834574895 2024-02-10T06:44:57.941 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-10T06:44:58.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.155+0000 7f26d8fe0700 1 -- 172.21.15.174:0/4028416530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4104750 msgr2=0x7f26d4104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.155+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/4028416530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4104750 0x7f26d4104bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b0a0 tx=0x7f26cc00b3b0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 -- 172.21.15.174:0/4028416530 shutdown_connections 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/4028416530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4104750 0x7f26d4104bd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/4028416530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d4106ce0 0x7f26d406a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/4028416530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26d406a7a0 0x7f26d406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 -- 172.21.15.174:0/4028416530 >> 172.21.15.174:0/4028416530 conn(0x7f26d4076c70 msgr2=0x7f26d4077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 -- 172.21.15.174:0/4028416530 shutdown_connections 2024-02-10T06:44:58.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.156+0000 7f26d8fe0700 1 -- 172.21.15.174:0/4028416530 wait complete. 2024-02-10T06:44:58.158 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 Processor -- start 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 -- start start 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d406a7a0 0x7f26d419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26d4104750 0x7f26d419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4106ce0 0x7f26d41a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d2d9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4106ce0 0x7f26d41a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d406a7a0 0x7f26d419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d2d9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4106ce0 0x7f26d41a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59466/0 (socket says 172.21.15.174:59466) 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d2d9d700 1 -- 172.21.15.174:0/945773417 learned_addr learned my addr 172.21.15.174:0/945773417 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41a2cb0 con 0x7f26d406a7a0 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41a2e20 con 0x7f26d4104750 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41a2fe0 con 0x7f26d4106ce0 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.157+0000 7f26d1d9b700 1 --2- 172.21.15.174:0/945773417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26d4104750 0x7f26d419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d2d9d700 1 -- 172.21.15.174:0/945773417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26d4104750 msgr2=0x7f26d419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:58.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d2d9d700 1 --2- 172.21.15.174:0/945773417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26d4104750 0x7f26d419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d2d9d700 1 -- 172.21.15.174:0/945773417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d406a7a0 msgr2=0x7f26d419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d2d9d700 1 --2- 172.21.15.174:0/945773417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d406a7a0 0x7f26d419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d2d9d700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009520 con 0x7f26d4106ce0 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d2d9d700 1 --2- 172.21.15.174:0/945773417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4106ce0 0x7f26d41a26a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26cc006130 tx=0x7f26cc004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00e040 con 0x7f26d4106ce0 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc009d00 con 0x7f26d4106ce0 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f26cc00be20 con 0x7f26d4106ce0 2024-02-10T06:44:58.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc005040 con 0x7f26d4106ce0 2024-02-10T06:44:58.161 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.158+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d41aa950 con 0x7f26d4106ce0 2024-02-10T06:44:58.161 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.159+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26d4117c30 con 0x7f26d4106ce0 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.160+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f26cc01d470 con 0x7f26d4106ce0 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26c37fe700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6800/1252702393 conn(0x7f26bc06da80 0x7f26bc06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f26cc08f0a0 con 0x7f26d4106ce0 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26d259c700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6800/1252702393 conn(0x7f26bc06da80 0x7f26bc06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26c37fe700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6809/2318790055 conn(0x7f26bc077420 0x7f26bc0798c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 --> v2:172.21.15.174:6809/2318790055 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26bc079f50 con 0x7f26bc077420 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc08f4d0 con 0x7f26d4106ce0 2024-02-10T06:44:58.162 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.161+0000 7f26d1d9b700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6809/2318790055 conn(0x7f26bc077420 0x7f26bc0798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:58.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.162+0000 7f26d259c700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6800/1252702393 conn(0x7f26bc06da80 0x7f26bc06ff40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009dc0 tx=0x7f26c4009260 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:58.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.162+0000 7f26d1d9b700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6809/2318790055 conn(0x7f26bc077420 0x7f26bc0798c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:58.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.162+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== osd.2 v2:172.21.15.174:6809/2318790055 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f26bc079f50 con 0x7f26bc077420 2024-02-10T06:44:58.180 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.179+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 --> v2:172.21.15.174:6809/2318790055 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26d41040a0 con 0x7f26bc077420 2024-02-10T06:44:58.180 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:57 smithi174 ceph-mon[29878]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-10T06:44:58.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:57 smithi174 ceph-mon[39523]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-10T06:44:58.181 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26c37fe700 1 -- 172.21.15.174:0/945773417 <== osd.2 v2:172.21.15.174:6809/2318790055 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26d41040a0 con 0x7f26bc077420 2024-02-10T06:44:58.181 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6809/2318790055 conn(0x7f26bc077420 msgr2=0x7f26bc0798c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:58.181 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6809/2318790055 conn(0x7f26bc077420 0x7f26bc0798c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:58.181 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6800/1252702393 conn(0x7f26bc06da80 msgr2=0x7f26bc06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/945773417 >> v2:172.21.15.174:6800/1252702393 conn(0x7f26bc06da80 0x7f26bc06ff40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009dc0 tx=0x7f26c4009260 comp rx=0 tx=0).stop 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4106ce0 msgr2=0x7f26d41a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 --2- 172.21.15.174:0/945773417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26d4106ce0 0x7f26d41a26a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26cc006130 tx=0x7f26cc004d00 comp rx=0 tx=0).stop 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d2d9d700 1 -- 172.21.15.174:0/945773417 reap_dead start 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 shutdown_connections 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.180+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 >> 172.21.15.174:0/945773417 conn(0x7f26d4076c70 msgr2=0x7f26d40ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.181+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 shutdown_connections 2024-02-10T06:44:58.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:58.181+0000 7f26d8fe0700 1 -- 172.21.15.174:0/945773417 wait complete. 2024-02-10T06:44:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:57 smithi184 ceph-mon[33513]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 -- 172.21.15.174:0/1349819146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8100c00 msgr2=0x7fccd8101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1349819146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8100c00 0x7fccd8101080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009990 tx=0x7fccc4009ca0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 -- 172.21.15.174:0/1349819146 shutdown_connections 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1349819146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd81015c0 0x7fccd810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1349819146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8106cb0 0x7fccd8107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1349819146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8100c00 0x7fccd8101080 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 -- 172.21.15.174:0/1349819146 >> 172.21.15.174:0/1349819146 conn(0x7fccd8077c30 msgr2=0x7fccd8078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 -- 172.21.15.174:0/1349819146 shutdown_connections 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.210+0000 7fccdef7d700 1 -- 172.21.15.174:0/1349819146 wait complete. 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 Processor -- start 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 -- start start 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8100c00 0x7fccd819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd81015c0 0x7fccd819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8106cb0 0x7fccd8196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8196c80 con 0x7fccd8100c00 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8196dc0 con 0x7fccd81015c0 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdef7d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8196f80 con 0x7fccd8106cb0 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8100c00 0x7fccd819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdd51a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8106cb0 0x7fccd8196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8100c00 0x7fccd819c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41108/0 (socket says 172.21.15.174:41108) 2024-02-10T06:44:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdd51a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8106cb0 0x7fccd8196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59474/0 (socket says 172.21.15.174:59474) 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 -- 172.21.15.174:0/1484132065 learned_addr learned my addr 172.21.15.174:0/1484132065 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccd7fff700 1 --2- 172.21.15.174:0/1484132065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd81015c0 0x7fccd819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 -- 172.21.15.174:0/1484132065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8106cb0 msgr2=0x7fccd8196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 --2- 172.21.15.174:0/1484132065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fccd8106cb0 0x7fccd8196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 -- 172.21.15.174:0/1484132065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd81015c0 msgr2=0x7fccd819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 --2- 172.21.15.174:0/1484132065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccd81015c0 0x7fccd819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.211+0000 7fccdcd19700 1 -- 172.21.15.174:0/1484132065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc009520 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.212+0000 7fccdcd19700 1 --2- 172.21.15.174:0/1484132065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8100c00 0x7fccd819c410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009f90 tx=0x7fcccc00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.212+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc009880 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.212+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fcccc0044d0 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.212+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc003e20 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.212+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc40095f0 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.212+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd806a440 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.213+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fccd8110690 con 0x7fccd8100c00 2024-02-10T06:44:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fcccc003f80 con 0x7fccd8100c00 2024-02-10T06:44:59.217 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd5ffb700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.174:6800/1252702393 conn(0x7fccc806da80 0x7fccc806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd7fff700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.174:6800/1252702393 conn(0x7fccc806da80 0x7fccc806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fcccc08f780 con 0x7fccd8100c00 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd5ffb700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.184:6800/3815653813 conn(0x7fccc80773a0 0x7fccc8079840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 --> v2:172.21.15.184:6800/3815653813 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccc8079ed0 con 0x7fccc80773a0 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcccc091e20 con 0x7fccd8100c00 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.214+0000 7fccd7fff700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.174:6800/1252702393 conn(0x7fccc806da80 0x7fccc806ff40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fccc400b7d0 tx=0x7fccc4005fd0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.215+0000 7fccdd51a700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.184:6800/3815653813 conn(0x7fccc80773a0 0x7fccc8079840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.215+0000 7fccdd51a700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.184:6800/3815653813 conn(0x7fccc80773a0 0x7fccc8079840 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.216+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== osd.4 v2:172.21.15.184:6800/3815653813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fccc8079ed0 con 0x7fccc80773a0 2024-02-10T06:44:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.232+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 --> v2:172.21.15.184:6800/3815653813 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fccd8106160 con 0x7fccc80773a0 2024-02-10T06:44:59.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccd5ffb700 1 -- 172.21.15.174:0/1484132065 <== osd.4 v2:172.21.15.184:6800/3815653813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fccd8106160 con 0x7fccc80773a0 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 >> v2:172.21.15.184:6800/3815653813 conn(0x7fccc80773a0 msgr2=0x7fccc8079840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.184:6800/3815653813 conn(0x7fccc80773a0 0x7fccc8079840 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 >> v2:172.21.15.174:6800/1252702393 conn(0x7fccc806da80 msgr2=0x7fccc806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1484132065 >> v2:172.21.15.174:6800/1252702393 conn(0x7fccc806da80 0x7fccc806ff40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fccc400b7d0 tx=0x7fccc4005fd0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8100c00 msgr2=0x7fccd819c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 --2- 172.21.15.174:0/1484132065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccd8100c00 0x7fccd819c410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009f90 tx=0x7fcccc00ede0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdd51a700 1 -- 172.21.15.174:0/1484132065 reap_dead start 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 shutdown_connections 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 >> 172.21.15.174:0/1484132065 conn(0x7fccd8077c30 msgr2=0x7fccd81047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 shutdown_connections 2024-02-10T06:44:59.235 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.233+0000 7fccdef7d700 1 -- 172.21.15.174:0/1484132065 wait complete. 2024-02-10T06:44:59.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 -- 172.21.15.174:0/956321238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c106cd0 msgr2=0x7f9d5c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/956321238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c106cd0 0x7f9d5c1070b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400d330 tx=0x7f9d5400d640 comp rx=0 tx=0).stop 2024-02-10T06:44:59.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 -- 172.21.15.174:0/956321238 shutdown_connections 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/956321238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c1015e0 0x7f9d5c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/956321238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c106cd0 0x7f9d5c1070b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/956321238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c100c20 0x7f9d5c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 -- 172.21.15.174:0/956321238 >> 172.21.15.174:0/956321238 conn(0x7f9d5c077d20 msgr2=0x7f9d5c078130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 -- 172.21.15.174:0/956321238 shutdown_connections 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.687+0000 7f9d60ebe700 1 -- 172.21.15.174:0/956321238 wait complete. 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 Processor -- start 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 -- start start 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c100c20 0x7f9d5c19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c1015e0 0x7f9d5c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c106cd0 0x7f9d5c195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c195e10 con 0x7f9d5c106cd0 2024-02-10T06:44:59.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c195f80 con 0x7f9d5c100c20 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d60ebe700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c196140 con 0x7f9d5c1015e0 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d59d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c1015e0 0x7f9d5c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d5ad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c106cd0 0x7f9d5c195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d5ad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c106cd0 0x7f9d5c195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41126/0 (socket says 172.21.15.174:41126) 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d5ad9d700 1 -- 172.21.15.174:0/1395289725 learned_addr learned my addr 172.21.15.174:0/1395289725 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.688+0000 7f9d5a59c700 1 --2- 172.21.15.174:0/1395289725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c100c20 0x7f9d5c19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d5ad9d700 1 -- 172.21.15.174:0/1395289725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c1015e0 msgr2=0x7f9d5c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d5ad9d700 1 --2- 172.21.15.174:0/1395289725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c1015e0 0x7f9d5c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d5ad9d700 1 -- 172.21.15.174:0/1395289725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c100c20 msgr2=0x7f9d5c19b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d5ad9d700 1 --2- 172.21.15.174:0/1395289725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c100c20 0x7f9d5c19b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d5ad9d700 1 -- 172.21.15.174:0/1395289725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009520 con 0x7f9d5c106cd0 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d59d9b700 1 --2- 172.21.15.174:0/1395289725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d5c1015e0 0x7f9d5c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d5ad9d700 1 --2- 172.21.15.174:0/1395289725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c106cd0 0x7f9d5c195660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00ea80 tx=0x7f9d4c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0098a0 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54009d00 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9d4c00c950 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c003c80 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.689+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c1a2510 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.690+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d5c10f510 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.690+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f9d4c004220 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d4b7fe700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406da80 0x7f9d4406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f9d4c014070 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d5a59c700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406da80 0x7f9d4406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d4b7fe700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6801/4183537518 conn(0x7f9d440773a0 0x7f9d44079840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 --> v2:172.21.15.174:6801/4183537518 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d44079ed0 con 0x7f9d440773a0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4c08fc70 con 0x7f9d5c106cd0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d59d9b700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6801/4183537518 conn(0x7f9d440773a0 0x7f9d44079840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.691+0000 7f9d5a59c700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406da80 0x7f9d4406ff40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400d280 tx=0x7f9d5401c040 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:59.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.692+0000 7f9d59d9b700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6801/4183537518 conn(0x7f9d440773a0 0x7f9d44079840 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:44:59.696 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.692+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== osd.0 v2:172.21.15.174:6801/4183537518 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9d44079ed0 con 0x7f9d440773a0 2024-02-10T06:44:59.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.711+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 --> v2:172.21.15.174:6801/4183537518 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d5c106180 con 0x7f9d440773a0 2024-02-10T06:44:59.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d4b7fe700 1 -- 172.21.15.174:0/1395289725 <== osd.0 v2:172.21.15.174:6801/4183537518 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d5c106180 con 0x7f9d440773a0 2024-02-10T06:44:59.713 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6801/4183537518 conn(0x7f9d440773a0 msgr2=0x7f9d44079840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.713 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6801/4183537518 conn(0x7f9d440773a0 0x7f9d44079840 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:44:59.713 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406da80 msgr2=0x7f9d4406ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/1395289725 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406da80 0x7f9d4406ff40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400d280 tx=0x7f9d5401c040 comp rx=0 tx=0).stop 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c106cd0 msgr2=0x7f9d5c195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 --2- 172.21.15.174:0/1395289725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5c106cd0 0x7f9d5c195660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00ea80 tx=0x7f9d4c00ed90 comp rx=0 tx=0).stop 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d5ad9d700 1 -- 172.21.15.174:0/1395289725 reap_dead start 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 shutdown_connections 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 >> 172.21.15.174:0/1395289725 conn(0x7f9d5c077d20 msgr2=0x7f9d5c104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 shutdown_connections 2024-02-10T06:44:59.714 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:44:59.712+0000 7f9d60ebe700 1 -- 172.21.15.174:0/1395289725 wait complete. 2024-02-10T06:45:00.138 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:44:59 smithi174 ceph-mon[29878]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:00.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:44:59 smithi174 ceph-mon[39523]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:00.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:44:59 smithi184 ceph-mon[33513]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:00.448 INFO:teuthology.orchestra.run.smithi174.stdout:34359738420 2024-02-10T06:45:00.449 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-10T06:45:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:00 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:00 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:01.373 INFO:teuthology.orchestra.run.smithi174.stdout:107374182437 2024-02-10T06:45:01.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-10T06:45:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:00 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:01 smithi174 ceph-mon[29878]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:02.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:01 smithi174 ceph-mon[39523]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:02.276 INFO:teuthology.orchestra.run.smithi174.stdout:128849018911 2024-02-10T06:45:02.276 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-10T06:45:02.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:01 smithi184 ceph-mon[33513]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 -- 172.21.15.174:0/536376852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0102730 msgr2=0x7fc8c010eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 --2- 172.21.15.174:0/536376852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0102730 0x7fc8c010eb70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b0a0 tx=0x7fc8b800b3b0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 -- 172.21.15.174:0/536376852 shutdown_connections 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 --2- 172.21.15.174:0/536376852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0102730 0x7fc8c010eb70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 --2- 172.21.15.174:0/536376852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0107d50 0x7fc8c0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 --2- 172.21.15.174:0/536376852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c0101d70 0x7fc8c01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 -- 172.21.15.174:0/536376852 >> 172.21.15.174:0/536376852 conn(0x7fc8c0078e50 msgr2=0x7fc8c0079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 -- 172.21.15.174:0/536376852 shutdown_connections 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.845+0000 7fc8c8336700 1 -- 172.21.15.174:0/536376852 wait complete. 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 Processor -- start 2024-02-10T06:45:02.383 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 -- start start 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0101d70 0x7fc8c0074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c0102730 0x7fc8c0074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0107d50 0x7fc8c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a5d70 con 0x7fc8c0101d70 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a5ee0 con 0x7fc8c0102730 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c8336700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a60a0 con 0x7fc8c0107d50 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0101d70 0x7fc8c0074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c68d3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0107d50 0x7fc8c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0101d70 0x7fc8c0074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41132/0 (socket says 172.21.15.174:41132) 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c68d3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0107d50 0x7fc8c0074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59514/0 (socket says 172.21.15.174:59514) 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 -- 172.21.15.174:0/2829980435 learned_addr learned my addr 172.21.15.174:0/2829980435 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c58d1700 1 --2- 172.21.15.174:0/2829980435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c0102730 0x7fc8c0074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 -- 172.21.15.174:0/2829980435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0107d50 msgr2=0x7fc8c0074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 --2- 172.21.15.174:0/2829980435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0107d50 0x7fc8c0074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 -- 172.21.15.174:0/2829980435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c0102730 msgr2=0x7fc8c0074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 --2- 172.21.15.174:0/2829980435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8c0102730 0x7fc8c0074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 -- 172.21.15.174:0/2829980435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc009520 con 0x7fc8c0101d70 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c68d3700 1 --2- 172.21.15.174:0/2829980435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c0107d50 0x7fc8c0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.846+0000 7fc8c60d2700 1 --2- 172.21.15.174:0/2829980435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0101d70 0x7fc8c0074020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00eb30 tx=0x7fc8bc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.847+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc009880 con 0x7fc8c0101d70 2024-02-10T06:45:02.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.847+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b8009d00 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.847+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c01a6740 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.847+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fc8bc0044d0 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.847+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc003c80 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.848+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8a4000fb0 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.848+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fc8bc004020 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8b77fe700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.174:6800/1252702393 conn(0x7fc8ac06da80 0x7fc8ac06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fc8bc014070 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8c58d1700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.174:6800/1252702393 conn(0x7fc8ac06da80 0x7fc8ac06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8b77fe700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.184:6804/2462414067 conn(0x7fc8ac0773a0 0x7fc8ac079840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 --> v2:172.21.15.184:6804/2462414067 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8ac079ed0 con 0x7fc8ac0773a0 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8bc08fc00 con 0x7fc8c0101d70 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8c58d1700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.174:6800/1252702393 conn(0x7fc8ac06da80 0x7fc8ac06ff40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0111f20 tx=0x7fc8b000a300 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.849+0000 7fc8c68d3700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.184:6804/2462414067 conn(0x7fc8ac0773a0 0x7fc8ac079840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.850+0000 7fc8c68d3700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.184:6804/2462414067 conn(0x7fc8ac0773a0 0x7fc8ac079840 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:02.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.851+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== osd.5 v2:172.21.15.184:6804/2462414067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc8ac079ed0 con 0x7fc8ac0773a0 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.867+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 --> v2:172.21.15.184:6804/2462414067 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc8a4002c90 con 0x7fc8ac0773a0 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8b77fe700 1 -- 172.21.15.174:0/2829980435 <== osd.5 v2:172.21.15.184:6804/2462414067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc8a4002c90 con 0x7fc8ac0773a0 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 >> v2:172.21.15.184:6804/2462414067 conn(0x7fc8ac0773a0 msgr2=0x7fc8ac079840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.184:6804/2462414067 conn(0x7fc8ac0773a0 0x7fc8ac079840 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 >> v2:172.21.15.174:6800/1252702393 conn(0x7fc8ac06da80 msgr2=0x7fc8ac06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 --2- 172.21.15.174:0/2829980435 >> v2:172.21.15.174:6800/1252702393 conn(0x7fc8ac06da80 0x7fc8ac06ff40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0111f20 tx=0x7fc8b000a300 comp rx=0 tx=0).stop 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0101d70 msgr2=0x7fc8c0074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 --2- 172.21.15.174:0/2829980435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c0101d70 0x7fc8c0074020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00eb30 tx=0x7fc8bc00ee40 comp rx=0 tx=0).stop 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c68d3700 1 -- 172.21.15.174:0/2829980435 reap_dead start 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 shutdown_connections 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.868+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 >> 172.21.15.174:0/2829980435 conn(0x7fc8c0078e50 msgr2=0x7fc8c0105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.869+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 shutdown_connections 2024-02-10T06:45:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.869+0000 7fc8c8336700 1 -- 172.21.15.174:0/2829980435 wait complete. 2024-02-10T06:45:02.970 INFO:teuthology.orchestra.run.smithi174.stdout:163208757275 2024-02-10T06:45:02.971 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-10T06:45:03.465 INFO:teuthology.orchestra.run.smithi174.stdout:77309411369 2024-02-10T06:45:03.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 -- 172.21.15.174:0/3252701709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 msgr2=0x7f28b8100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/3252701709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b8100fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28a80099c0 tx=0x7f28a8009cd0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 -- 172.21.15.174:0/3252701709 shutdown_connections 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/3252701709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8101500 0x7f28b810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/3252701709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 0x7f28b8106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/3252701709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b8100fc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 -- 172.21.15.174:0/3252701709 >> 172.21.15.174:0/3252701709 conn(0x7f28b80fc610 msgr2=0x7f28b80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 -- 172.21.15.174:0/3252701709 shutdown_connections 2024-02-10T06:45:03.590 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.957+0000 7f28bd7ff700 1 -- 172.21.15.174:0/3252701709 wait complete. 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 Processor -- start 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 -- start start 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8101500 0x7f28b8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 0x7f28b81a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81993d0 con 0x7f28b8100b40 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a3bb0 con 0x7f28b8101500 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28bd7ff700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b81a3cf0 con 0x7f28b8106bf0 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28b6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28b77fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 0x7f28b81a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28b6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41150/0 (socket says 172.21.15.174:41150) 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28b77fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 0x7f28b81a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59538/0 (socket says 172.21.15.174:59538) 2024-02-10T06:45:03.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.958+0000 7f28b6ffd700 1 -- 172.21.15.174:0/1645007580 learned_addr learned my addr 172.21.15.174:0/1645007580 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b67fc700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8101500 0x7f28b8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b6ffd700 1 -- 172.21.15.174:0/1645007580 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 msgr2=0x7f28b81a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b6ffd700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 0x7f28b81a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b6ffd700 1 -- 172.21.15.174:0/1645007580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8101500 msgr2=0x7f28b8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b6ffd700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8101500 0x7f28b8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b6ffd700 1 -- 172.21.15.174:0/1645007580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac009520 con 0x7f28b8100b40 2024-02-10T06:45:03.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b77fe700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28b8106bf0 0x7f28b81a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b67fc700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28b8101500 0x7f28b8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f28b6ffd700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b81985f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00eb60 tx=0x7f28ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac00cc90 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.959+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f28ac0044d0 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.960+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac003e90 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.960+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a80095f0 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.960+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b81967d0 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.960+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28b81105a0 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.961+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f28ac018430 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.961+0000 7f289ffff700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.174:6800/1252702393 conn(0x7f28a006da80 0x7f28a006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.961+0000 7f28b67fc700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.174:6800/1252702393 conn(0x7f28a006da80 0x7f28a006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.961+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f28ac014070 con 0x7f28b8100b40 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.962+0000 7f289ffff700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.184:6812/3506789608 conn(0x7f28a00772b0 0x7f28a0079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.962+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 --> v2:172.21.15.184:6812/3506789608 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28a0079de0 con 0x7f28a00772b0 2024-02-10T06:45:03.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.962+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28ac08fc20 con 0x7f28b8100b40 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.962+0000 7f28b67fc700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.174:6800/1252702393 conn(0x7f28a006da80 0x7f28a006ff40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f28a8000c00 tx=0x7f28a8005ab0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.962+0000 7f28b77fe700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.184:6812/3506789608 conn(0x7f28a00772b0 0x7f28a0079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.962+0000 7f28b77fe700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.184:6812/3506789608 conn(0x7f28a00772b0 0x7f28a0079750 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.963+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== osd.7 v2:172.21.15.184:6812/3506789608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f28a0079de0 con 0x7f28a00772b0 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.981+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 --> v2:172.21.15.184:6812/3506789608 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28b8106060 con 0x7f28a00772b0 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.981+0000 7f289ffff700 1 -- 172.21.15.174:0/1645007580 <== osd.7 v2:172.21.15.184:6812/3506789608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f28b8106060 con 0x7f28a00772b0 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.981+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 >> v2:172.21.15.184:6812/3506789608 conn(0x7f28a00772b0 msgr2=0x7f28a0079750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.981+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.184:6812/3506789608 conn(0x7f28a00772b0 0x7f28a0079750 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 >> v2:172.21.15.174:6800/1252702393 conn(0x7f28a006da80 msgr2=0x7f28a006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/1645007580 >> v2:172.21.15.174:6800/1252702393 conn(0x7f28a006da80 0x7f28a006ff40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f28a8000c00 tx=0x7f28a8005ab0 comp rx=0 tx=0).stop 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 msgr2=0x7f28b81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 --2- 172.21.15.174:0/1645007580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b8100b40 0x7f28b81985f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00eb60 tx=0x7f28ac00ee70 comp rx=0 tx=0).stop 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28b77fe700 1 -- 172.21.15.174:0/1645007580 reap_dead start 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 shutdown_connections 2024-02-10T06:45:03.594 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 >> 172.21.15.174:0/1645007580 conn(0x7f28b80fc610 msgr2=0x7f28b81046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:03.595 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 shutdown_connections 2024-02-10T06:45:03.595 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:00.982+0000 7f28bd7ff700 1 -- 172.21.15.174:0/1645007580 wait complete. 2024-02-10T06:45:03.779 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:04.131 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:03 smithi174 ceph-mon[29878]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:04.132 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:03 smithi174 ceph-mon[39523]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:04.132 INFO:teuthology.orchestra.run.smithi174.stdout:231928234001 2024-02-10T06:45:04.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-10T06:45:04.175 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:03 smithi184 ceph-mon[33513]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T06:45:04.991 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:04.992 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:04.993 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:04.996 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:06.039 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:05 smithi174 ceph-mon[29878]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:06.040 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:05 smithi174 ceph-mon[39523]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:05 smithi184 ceph-mon[33513]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:06.374 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:45:05 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:45:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:07.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:45:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:45:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:08 smithi174 ceph-mon[29878]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:08 smithi174 ceph-mon[39523]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:08.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:08 smithi184 ceph-mon[33513]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:09.637 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:09.637 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 -- 172.21.15.174:0/1704147584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 msgr2=0x7f49d8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 --2- 172.21.15.174:0/1704147584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8105820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009990 tx=0x7f49c4009ca0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 -- 172.21.15.174:0/1704147584 shutdown_connections 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 --2- 172.21.15.174:0/1704147584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d8105f00 0x7f49d810dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 --2- 172.21.15.174:0/1704147584 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8100eb0 0x7f49d8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 --2- 172.21.15.174:0/1704147584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8105820 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 -- 172.21.15.174:0/1704147584 >> 172.21.15.174:0/1704147584 conn(0x7f49d8076c70 msgr2=0x7f49d8077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 -- 172.21.15.174:0/1704147584 shutdown_connections 2024-02-10T06:45:09.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.643+0000 7f49dec65700 1 -- 172.21.15.174:0/1704147584 wait complete. 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.644+0000 7f49dec65700 1 Processor -- start 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.644+0000 7f49dec65700 1 -- start start 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.644+0000 7f49dec65700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d8100eb0 0x7f49d8195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.644+0000 7f49dec65700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dec65700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8105f00 0x7f49d819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dec65700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d819a620 con 0x7f49d8101860 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dec65700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d819a790 con 0x7f49d8100eb0 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dec65700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d819a950 con 0x7f49d8105f00 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56662/0 (socket says 172.21.15.174:56662) 2024-02-10T06:45:09.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8105f00 0x7f49d819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49d7fff700 1 -- 172.21.15.174:0/3790301791 learned_addr learned my addr 172.21.15.174:0/3790301791 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dca01700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d8100eb0 0x7f49d8195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 -- 172.21.15.174:0/3790301791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d8100eb0 msgr2=0x7f49d8195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d8100eb0 0x7f49d8195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 -- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 msgr2=0x7f49d8195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 -- 172.21.15.174:0/3790301791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40095f0 con 0x7f49d8105f00 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49d7fff700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:09.647 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.645+0000 7f49dd202700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8105f00 0x7f49d819a010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49d0011180 tx=0x7f49d0011490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:09.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.646+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0004bb0 con 0x7f49d8105f00 2024-02-10T06:45:09.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.646+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d819ac30 con 0x7f49d8105f00 2024-02-10T06:45:09.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.646+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f49d0013040 con 0x7f49d8105f00 2024-02-10T06:45:09.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.646+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0019660 con 0x7f49d8105f00 2024-02-10T06:45:09.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.646+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d81a2490 con 0x7f49d8105f00 2024-02-10T06:45:09.648 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.647+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8109220 con 0x7f49d8105f00 2024-02-10T06:45:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.648+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f49d0005020 con 0x7f49d8105f00 2024-02-10T06:45:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.648+0000 7f49d5ffb700 1 --2- 172.21.15.174:0/3790301791 >> v2:172.21.15.174:6800/1252702393 conn(0x7f49c806da80 0x7f49c806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.649+0000 7f49dca01700 1 --2- 172.21.15.174:0/3790301791 >> v2:172.21.15.174:6800/1252702393 conn(0x7f49c806da80 0x7f49c806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.649+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f49d0090f70 con 0x7f49d8105f00 2024-02-10T06:45:09.650 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.649+0000 7f49dca01700 1 --2- 172.21.15.174:0/3790301791 >> v2:172.21.15.174:6800/1252702393 conn(0x7f49c806da80 0x7f49c806ff40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009bf0 tx=0x7f49cc009260 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:09.655 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.654+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49d00c4d20 con 0x7f49d8105f00 2024-02-10T06:45:09.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.857+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f49d8196aa0 con 0x7f49d8105f00 2024-02-10T06:45:09.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.857+0000 7f49d5ffb700 1 -- 172.21.15.174:0/3790301791 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49d005e400 con 0x7f49d8105f00 2024-02-10T06:45:09.858 INFO:teuthology.orchestra.run.smithi174.stdout:55834574897 2024-02-10T06:45:09.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.859+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 >> v2:172.21.15.174:6800/1252702393 conn(0x7f49c806da80 msgr2=0x7f49c806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:09.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.859+0000 7f49dec65700 1 --2- 172.21.15.174:0/3790301791 >> v2:172.21.15.174:6800/1252702393 conn(0x7f49c806da80 0x7f49c806ff40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009bf0 tx=0x7f49cc009260 comp rx=0 tx=0).stop 2024-02-10T06:45:09.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.859+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8105f00 msgr2=0x7f49d819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.859+0000 7f49dec65700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8105f00 0x7f49d819a010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49d0011180 tx=0x7f49d0011490 comp rx=0 tx=0).stop 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 shutdown_connections 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49d8105f00 0x7f49d819a010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 --2- 172.21.15.174:0/3790301791 >> v2:172.21.15.174:6800/1252702393 conn(0x7f49c806da80 0x7f49c806ff40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49d8100eb0 0x7f49d8195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 --2- 172.21.15.174:0/3790301791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49d8101860 0x7f49d8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 >> 172.21.15.174:0/3790301791 conn(0x7f49d8076c70 msgr2=0x7f49d80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 shutdown_connections 2024-02-10T06:45:09.861 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:09.860+0000 7f49dec65700 1 -- 172.21.15.174:0/3790301791 wait complete. 2024-02-10T06:45:10.685 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:10 smithi174 ceph-mon[29878]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:10.685 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3790301791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T06:45:10.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:10 smithi174 ceph-mon[39523]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:10.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3790301791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T06:45:10.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:10 smithi184 ceph-mon[33513]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3790301791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T06:45:11.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 -- 172.21.15.174:0/318130618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08105f80 msgr2=0x7f4e0810db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 --2- 172.21.15.174:0/318130618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08105f80 0x7f4e0810db50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009f90 tx=0x7f4e0000b330 comp rx=0 tx=0).stop 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 -- 172.21.15.174:0/318130618 shutdown_connections 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 --2- 172.21.15.174:0/318130618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08105f80 0x7f4e0810db50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 --2- 172.21.15.174:0/318130618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 0x7f4e08101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 --2- 172.21.15.174:0/318130618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08101850 0x7f4e081058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 -- 172.21.15.174:0/318130618 >> 172.21.15.174:0/318130618 conn(0x7f4e08077c80 msgr2=0x7f4e08078090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 -- 172.21.15.174:0/318130618 shutdown_connections 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 -- 172.21.15.174:0/318130618 wait complete. 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.882+0000 7f4e0d898700 1 Processor -- start 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 -- start start 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 0x7f4e0819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 0x7f4e0819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08105f80 0x7f4e08196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:11.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08196eb0 con 0x7f4e08101850 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08197020 con 0x7f4e08105f80 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e0d898700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081971e0 con 0x7f4e08100ea0 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e067fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 0x7f4e0819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e06ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 0x7f4e0819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e067fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 0x7f4e0819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56692/0 (socket says 172.21.15.174:56692) 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e067fc700 1 -- 172.21.15.174:0/4058281105 learned_addr learned my addr 172.21.15.174:0/4058281105 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e077fe700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08105f80 0x7f4e08196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:11.121 INFO:teuthology.orchestra.run.smithi174.stdout:107374182439 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.883+0000 7f4e077fe700 1 -- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 msgr2=0x7f4e0819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e077fe700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 0x7f4e0819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e077fe700 1 -- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 msgr2=0x7f4e0819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e077fe700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 0x7f4e0819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e077fe700 1 -- 172.21.15.174:0/4058281105 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80095f0 con 0x7f4e08105f80 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e067fc700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 0x7f4e0819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e06ffd700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 0x7f4e0819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e077fe700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08105f80 0x7f4e08196840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b6c0 tx=0x7f4e00009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000e040 con 0x7f4e08105f80 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00009c40 con 0x7f4e08105f80 2024-02-10T06:45:11.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f4e0000be30 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000052a0 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.884+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e081a3500 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.885+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e08109240 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.886+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f4e0001d470 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.887+0000 7f4e0c896700 1 --2- 172.21.15.174:0/4058281105 >> v2:172.21.15.174:6800/1252702393 conn(0x7f4df006da80 0x7f4df006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.887+0000 7f4e06ffd700 1 --2- 172.21.15.174:0/4058281105 >> v2:172.21.15.174:6800/1252702393 conn(0x7f4df006da80 0x7f4df006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.887+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f4e0008e6d0 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.887+0000 7f4e06ffd700 1 --2- 172.21.15.174:0/4058281105 >> v2:172.21.15.174:6800/1252702393 conn(0x7f4df006da80 0x7f4df006ff40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005fd0 tx=0x7f4df8009310 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:10.892+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e0005bae0 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.099+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4e08197d50 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.100+0000 7f4e0c896700 1 -- 172.21.15.174:0/4058281105 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e0005b670 con 0x7f4e08105f80 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 >> v2:172.21.15.174:6800/1252702393 conn(0x7f4df006da80 msgr2=0x7f4df006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 --2- 172.21.15.174:0/4058281105 >> v2:172.21.15.174:6800/1252702393 conn(0x7f4df006da80 0x7f4df006ff40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005fd0 tx=0x7f4df8009310 comp rx=0 tx=0).stop 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08105f80 msgr2=0x7f4e08196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08105f80 0x7f4e08196840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b6c0 tx=0x7f4e00009420 comp rx=0 tx=0).stop 2024-02-10T06:45:11.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 shutdown_connections 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 --2- 172.21.15.174:0/4058281105 >> v2:172.21.15.174:6800/1252702393 conn(0x7f4df006da80 0x7f4df006ff40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e08105f80 0x7f4e08196840 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e08100ea0 0x7f4e0819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 --2- 172.21.15.174:0/4058281105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e08101850 0x7f4e0819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 >> 172.21.15.174:0/4058281105 conn(0x7f4e08077c80 msgr2=0x7f4e080fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 shutdown_connections 2024-02-10T06:45:11.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:11.102+0000 7f4e0d898700 1 -- 172.21.15.174:0/4058281105 wait complete. 2024-02-10T06:45:11.706 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4058281105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T06:45:11.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4058281105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T06:45:11.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182437 got 107374182439 for osd.3 2024-02-10T06:45:11.734 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4058281105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T06:45:12.774 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:12 smithi174 ceph-mon[29878]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:45:12.774 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:12 smithi174 ceph-mon[39523]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:45:12.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:12 smithi184 ceph-mon[33513]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:45:13.137 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:13.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574895 got 55834574897 for osd.1 2024-02-10T06:45:13.173 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:13.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 -- 172.21.15.174:0/1589267482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3941015c0 msgr2=0x7fb39410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:13.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 --2- 172.21.15.174:0/1589267482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3941015c0 0x7fb39410daa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b0a0 tx=0x7fb38c00b3b0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 -- 172.21.15.174:0/1589267482 shutdown_connections 2024-02-10T06:45:13.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 --2- 172.21.15.174:0/1589267482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3941015c0 0x7fb39410daa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 --2- 172.21.15.174:0/1589267482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 0x7fb394107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 --2- 172.21.15.174:0/1589267482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb394100c20 0x7fb394101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 -- 172.21.15.174:0/1589267482 >> 172.21.15.174:0/1589267482 conn(0x7fb394077b50 msgr2=0x7fb394077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 -- 172.21.15.174:0/1589267482 shutdown_connections 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.391+0000 7fb39a098700 1 -- 172.21.15.174:0/1589267482 wait complete. 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 Processor -- start 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 -- start start 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb394100c20 0x7fb394198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb3941015c0 0x7fb394198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 0x7fb3941a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb394199640 con 0x7fb394100c20 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a3e50 con 0x7fb394106c60 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.392+0000 7fb39a098700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941a3f90 con 0x7fb3941015c0 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb394100c20 0x7fb394198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb392ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb3941015c0 0x7fb394198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb394100c20 0x7fb394198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56710/0 (socket says 172.21.15.174:56710) 2024-02-10T06:45:13.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb3937fe700 1 -- 172.21.15.174:0/1076881271 learned_addr learned my addr 172.21.15.174:0/1076881271 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 0x7fb3941a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 -- 172.21.15.174:0/1076881271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb3941015c0 msgr2=0x7fb394198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb3941015c0 0x7fb394198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 -- 172.21.15.174:0/1076881271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb394100c20 msgr2=0x7fb394198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb394100c20 0x7fb394198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 -- 172.21.15.174:0/1076881271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388009520 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb393fff700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 0x7fb3941a3910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b070 tx=0x7fb38c00bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00e040 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fb38c004e60 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c009d00 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01dd90 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.393+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb394196960 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.394+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb394109180 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.395+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fb38c005040 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.395+0000 7fb390ff9700 1 --2- 172.21.15.174:0/1076881271 >> v2:172.21.15.174:6800/1252702393 conn(0x7fb37c06da80 0x7fb37c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.395+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fb38c08e760 con 0x7fb394106c60 2024-02-10T06:45:13.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.395+0000 7fb3937fe700 1 --2- 172.21.15.174:0/1076881271 >> v2:172.21.15.174:6800/1252702393 conn(0x7fb37c06da80 0x7fb37c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:13.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.396+0000 7fb3937fe700 1 --2- 172.21.15.174:0/1076881271 >> v2:172.21.15.174:6800/1252702393 conn(0x7fb37c06da80 0x7fb37c06ff40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb384009ca0 tx=0x7fb384009260 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:13.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.400+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb38c05baf0 con 0x7fb394106c60 2024-02-10T06:45:13.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.610+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb394069f90 con 0x7fb394106c60 2024-02-10T06:45:13.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.610+0000 7fb390ff9700 1 -- 172.21.15.174:0/1076881271 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb38c05b680 con 0x7fb394106c60 2024-02-10T06:45:13.611 INFO:teuthology.orchestra.run.smithi174.stdout:34359738423 2024-02-10T06:45:13.613 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 >> v2:172.21.15.174:6800/1252702393 conn(0x7fb37c06da80 msgr2=0x7fb37c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:13.613 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 --2- 172.21.15.174:0/1076881271 >> v2:172.21.15.174:6800/1252702393 conn(0x7fb37c06da80 0x7fb37c06ff40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb384009ca0 tx=0x7fb384009260 comp rx=0 tx=0).stop 2024-02-10T06:45:13.613 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 msgr2=0x7fb3941a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 0x7fb3941a3910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b070 tx=0x7fb38c00bdd0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 shutdown_connections 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 --2- 172.21.15.174:0/1076881271 >> v2:172.21.15.174:6800/1252702393 conn(0x7fb37c06da80 0x7fb37c06ff40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb394106c60 0x7fb3941a3910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb394100c20 0x7fb394198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 --2- 172.21.15.174:0/1076881271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb3941015c0 0x7fb394198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 >> 172.21.15.174:0/1076881271 conn(0x7fb394077b50 msgr2=0x7fb3940ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 shutdown_connections 2024-02-10T06:45:13.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:13.613+0000 7fb39a098700 1 -- 172.21.15.174:0/1076881271 wait complete. 2024-02-10T06:45:13.658 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.658 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.658 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.658 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.658 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi174 to 3760M 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1076881271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.659 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi174 to 3760M 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1076881271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:13.660 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:13 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi174 to 3760M 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: Updating smithi174:/etc/ceph/ceph.conf 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: Updating smithi184:/etc/ceph/ceph.conf 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1076881271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T06:45:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:13 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.620 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.185+0000 7fa288f7e700 1 -- 172.21.15.174:0/1653667009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2841008c0 msgr2=0x7fa284100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:14.620 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.185+0000 7fa288f7e700 1 --2- 172.21.15.174:0/1653667009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2841008c0 0x7fa284100ca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2740099c0 tx=0x7fa274009cd0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.620 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 -- 172.21.15.174:0/1653667009 shutdown_connections 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 --2- 172.21.15.174:0/1653667009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa284105790 0x7fa28410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 --2- 172.21.15.174:0/1653667009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2841008c0 0x7fa284100ca0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 --2- 172.21.15.174:0/1653667009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa284101270 0x7fa284105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 -- 172.21.15.174:0/1653667009 >> 172.21.15.174:0/1653667009 conn(0x7fa284075f60 msgr2=0x7fa284076370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 -- 172.21.15.174:0/1653667009 shutdown_connections 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 -- 172.21.15.174:0/1653667009 wait complete. 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 Processor -- start 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.186+0000 7fa288f7e700 1 -- start start 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2841008c0 0x7fa28419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 0x7fa28419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa284105790 0x7fa284196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284196e20 con 0x7fa284101270 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284196f90 con 0x7fa284105790 2024-02-10T06:45:14.621 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa284197150 con 0x7fa2841008c0 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 0x7fa28419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa28259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2841008c0 0x7fa28419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 0x7fa28419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56736/0 (socket says 172.21.15.174:56736) 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 -- 172.21.15.174:0/3398798090 learned_addr learned my addr 172.21.15.174:0/3398798090 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa282d9d700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa284105790 0x7fa284196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 -- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2841008c0 msgr2=0x7fa28419c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2841008c0 0x7fa28419c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 -- 172.21.15.174:0/3398798090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa284105790 msgr2=0x7fa284196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa284105790 0x7fa284196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 -- 172.21.15.174:0/3398798090 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa278009520 con 0x7fa284101270 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa28259c700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2841008c0 0x7fa28419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa281d9b700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 0x7fa28419cae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa278009fd0 tx=0x7fa27800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:14.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27800cbd0 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stdout:197568495640 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2740095f0 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa27800eea0 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278003c80 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.187+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2841a3520 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.188+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa284101ad0 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.189+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fa278003e40 con 0x7fa284101270 2024-02-10T06:45:14.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.189+0000 7fa2737fe700 1 --2- 172.21.15.174:0/3398798090 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa26c06da80 0x7fa26c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.189+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fa278014070 con 0x7fa284101270 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.190+0000 7fa28259c700 1 --2- 172.21.15.174:0/3398798090 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa26c06da80 0x7fa26c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.190+0000 7fa28259c700 1 --2- 172.21.15.174:0/3398798090 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa26c06da80 0x7fa26c06ff40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa274000c00 tx=0x7fa274005fb0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.194+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa27805aa30 con 0x7fa284101270 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.394+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa284197ba0 con 0x7fa284101270 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.394+0000 7fa2737fe700 1 -- 172.21.15.174:0/3398798090 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa27805a5c0 con 0x7fa284101270 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa26c06da80 msgr2=0x7fa26c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 --2- 172.21.15.174:0/3398798090 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa26c06da80 0x7fa26c06ff40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa274000c00 tx=0x7fa274005fb0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 msgr2=0x7fa28419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 0x7fa28419cae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa278009fd0 tx=0x7fa27800ed10 comp rx=0 tx=0).stop 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 shutdown_connections 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 --2- 172.21.15.174:0/3398798090 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa26c06da80 0x7fa26c06ff40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa284105790 0x7fa284196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2841008c0 0x7fa28419c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 --2- 172.21.15.174:0/3398798090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101270 0x7fa28419cae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:14.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 >> 172.21.15.174:0/3398798090 conn(0x7fa284075f60 msgr2=0x7fa284103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:14.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 shutdown_connections 2024-02-10T06:45:14.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:14.396+0000 7fa288f7e700 1 -- 172.21.15.174:0/3398798090 wait complete. 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: Deploying daemon alertmanager.a on smithi174 2024-02-10T06:45:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3398798090' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: Deploying daemon alertmanager.a on smithi174 2024-02-10T06:45:14.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3398798090' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T06:45:15.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Updating smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Updating smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: Deploying daemon alertmanager.a on smithi174 2024-02-10T06:45:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3398798090' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T06:45:15.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:15 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:15.894 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:15 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:15 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:16.375 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:45:15 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:45:16.422 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738420 got 34359738423 for osd.0 2024-02-10T06:45:16.423 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:16.664 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:16 smithi174 ceph-mon[29878]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:16 smithi174 ceph-mon[39523]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:17.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:16 smithi184 ceph-mon[33513]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:17.397 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495636 got 197568495640 for osd.6 2024-02-10T06:45:17.397 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:17.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:17.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:45:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:45:17.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:18.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 -- 172.21.15.174:0/234095597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc101e30 msgr2=0x7f27dc1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/234095597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc101e30 0x7f27dc1022b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009990 tx=0x7f27d0009ca0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 -- 172.21.15.174:0/234095597 shutdown_connections 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/234095597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc1027f0 0x7f27dc10eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/234095597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc107df0 0x7f27dc1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/234095597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc101e30 0x7f27dc1022b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 -- 172.21.15.174:0/234095597 >> 172.21.15.174:0/234095597 conn(0x7f27dc078e80 msgr2=0x7f27dc079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 -- 172.21.15.174:0/234095597 shutdown_connections 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.564+0000 7f27e0f3b700 1 -- 172.21.15.174:0/234095597 wait complete. 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 Processor -- start 2024-02-10T06:45:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 -- start start 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc101e30 0x7f27dc074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc1027f0 0x7f27dc074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 0x7f27dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1a5c30 con 0x7f27dc107df0 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1a5da0 con 0x7f27dc101e30 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27e0f3b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1a5f60 con 0x7f27dc1027f0 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27d9d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc1027f0 0x7f27dc074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 0x7f27dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 0x7f27dc074aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:59996/0 (socket says 172.21.15.174:59996) 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27d9d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc1027f0 0x7f27dc074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:42592/0 (socket says 172.21.15.174:42592) 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 -- 172.21.15.174:0/992119679 learned_addr learned my addr 172.21.15.174:0/992119679 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27da59c700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc101e30 0x7f27dc074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 -- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc1027f0 msgr2=0x7f27dc074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc1027f0 0x7f27dc074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 -- 172.21.15.174:0/992119679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc101e30 msgr2=0x7f27dc074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc101e30 0x7f27dc074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.565+0000 7f27dad9d700 1 -- 172.21.15.174:0/992119679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4009c40 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27da59c700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc101e30 0x7f27dc074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27dad9d700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 0x7f27dc074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c370 tx=0x7f27d400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4017070 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f27d400f040 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400e050 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d00095f0 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.566+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc1a64d0 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.567+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc10a140 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.567+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f27d4004470 con 0x7f27dc107df0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.568+0000 7f27cb7fe700 1 --2- 172.21.15.174:0/992119679 >> v2:172.21.15.174:6800/1252702393 conn(0x7f27c406da80 0x7f27c406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.568+0000 7f27da59c700 1 --2- 172.21.15.174:0/992119679 >> v2:172.21.15.174:6800/1252702393 conn(0x7f27c406da80 0x7f27c406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.579 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.568+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f27d408cd60 con 0x7f27dc107df0 2024-02-10T06:45:18.580 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.568+0000 7f27da59c700 1 --2- 172.21.15.174:0/992119679 >> v2:172.21.15.174:6800/1252702393 conn(0x7f27c406da80 0x7f27c406ff40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00baa0 tx=0x7f27cc005db0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:18.580 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.572+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27d405a170 con 0x7f27dc107df0 2024-02-10T06:45:18.729 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:18 smithi174 ceph-mon[29878]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:18.730 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:18 smithi174 ceph-mon[39523]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:18.787 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.786+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f27dc103430 con 0x7f27dc107df0 2024-02-10T06:45:18.787 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.786+0000 7f27cb7fe700 1 -- 172.21.15.174:0/992119679 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f27d4059d00 con 0x7f27dc107df0 2024-02-10T06:45:18.787 INFO:teuthology.orchestra.run.smithi174.stdout:163208757279 2024-02-10T06:45:18.789 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 >> v2:172.21.15.174:6800/1252702393 conn(0x7f27c406da80 msgr2=0x7f27c406ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.789 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/992119679 >> v2:172.21.15.174:6800/1252702393 conn(0x7f27c406da80 0x7f27c406ff40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00baa0 tx=0x7f27cc005db0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 msgr2=0x7f27dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 0x7f27dc074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c370 tx=0x7f27d400c680 comp rx=0 tx=0).stop 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 shutdown_connections 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27dc107df0 0x7f27dc074aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/992119679 >> v2:172.21.15.174:6800/1252702393 conn(0x7f27c406da80 0x7f27c406ff40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27dc101e30 0x7f27dc074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 --2- 172.21.15.174:0/992119679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27dc1027f0 0x7f27dc074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 >> 172.21.15.174:0/992119679 conn(0x7f27dc078e80 msgr2=0x7f27dc100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 shutdown_connections 2024-02-10T06:45:18.790 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.789+0000 7f27e0f3b700 1 -- 172.21.15.174:0/992119679 wait complete. 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 -- 172.21.15.174:0/3983500652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050107db0 msgr2=0x7fe050108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 --2- 172.21.15.174:0/3983500652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050107db0 0x7fe050108190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009990 tx=0x7fe04c009ca0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 -- 172.21.15.174:0/3983500652 shutdown_connections 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 --2- 172.21.15.174:0/3983500652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe050102740 0x7fe05010ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 --2- 172.21.15.174:0/3983500652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050107db0 0x7fe050108190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 --2- 172.21.15.174:0/3983500652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050101da0 0x7fe050102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 -- 172.21.15.174:0/3983500652 >> 172.21.15.174:0/3983500652 conn(0x7fe050078df0 msgr2=0x7fe050079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 -- 172.21.15.174:0/3983500652 shutdown_connections 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 -- 172.21.15.174:0/3983500652 wait complete. 2024-02-10T06:45:18.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 Processor -- start 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 -- start start 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.841+0000 7fe0586b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe050101da0 0x7fe05019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050102740 0x7fe05019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 0x7fe0501977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe050197e80 con 0x7fe050107db0 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe050197ff0 con 0x7fe050101da0 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0501981b0 con 0x7fe050102740 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 0x7fe0501977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 0x7fe0501977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:60008/0 (socket says 172.21.15.174:60008) 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 -- 172.21.15.174:0/2854752403 learned_addr learned my addr 172.21.15.174:0/2854752403 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe055c4f700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050102740 0x7fe05019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056450700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe050101da0 0x7fe05019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 -- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050102740 msgr2=0x7fe05019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050102740 0x7fe05019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 -- 172.21.15.174:0/2854752403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe050101da0 msgr2=0x7fe05019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe050101da0 0x7fe05019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 -- 172.21.15.174:0/2854752403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040009520 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe056c51700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 0x7fe0501977d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe048007ee0 tx=0x7fe04800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048017070 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fe04800f040 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0095f0 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048005730 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.842+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0501a4460 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.844+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05010a2b0 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.844+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fe04800dbd0 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.844+0000 7fe0477fe700 1 --2- 172.21.15.174:0/2854752403 >> v2:172.21.15.174:6800/1252702393 conn(0x7fe03c06da80 0x7fe03c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.844+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fe04808cc30 con 0x7fe050107db0 2024-02-10T06:45:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.844+0000 7fe056450700 1 --2- 172.21.15.174:0/2854752403 >> v2:172.21.15.174:6800/1252702393 conn(0x7fe03c06da80 0x7fe03c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:18.891 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.845+0000 7fe056450700 1 --2- 172.21.15.174:0/2854752403 >> v2:172.21.15.174:6800/1252702393 conn(0x7fe03c06da80 0x7fe03c06ff40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00b7d0 tx=0x7fe04c000bc0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:18.891 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:18.849+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe04805a040 con 0x7fe050107db0 2024-02-10T06:45:19.084 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.084+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe0500689e0 con 0x7fe050107db0 2024-02-10T06:45:19.085 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.084+0000 7fe0477fe700 1 -- 172.21.15.174:0/2854752403 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe048059bd0 con 0x7fe050107db0 2024-02-10T06:45:19.085 INFO:teuthology.orchestra.run.smithi174.stdout:128849018915 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 >> v2:172.21.15.174:6800/1252702393 conn(0x7fe03c06da80 msgr2=0x7fe03c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 --2- 172.21.15.174:0/2854752403 >> v2:172.21.15.174:6800/1252702393 conn(0x7fe03c06da80 0x7fe03c06ff40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00b7d0 tx=0x7fe04c000bc0 comp rx=0 tx=0).stop 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 msgr2=0x7fe0501977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 0x7fe0501977d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe048007ee0 tx=0x7fe04800d3b0 comp rx=0 tx=0).stop 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 shutdown_connections 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe050107db0 0x7fe0501977d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:19.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 --2- 172.21.15.174:0/2854752403 >> v2:172.21.15.174:6800/1252702393 conn(0x7fe03c06da80 0x7fe03c06ff40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:19.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe050101da0 0x7fe05019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:19.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 --2- 172.21.15.174:0/2854752403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe050102740 0x7fe05019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:19.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 >> 172.21.15.174:0/2854752403 conn(0x7fe050078df0 msgr2=0x7fe050100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:19.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 shutdown_connections 2024-02-10T06:45:19.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:19.086+0000 7fe0586b4700 1 -- 172.21.15.174:0/2854752403 wait complete. 2024-02-10T06:45:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:18 smithi184 ceph-mon[33513]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/992119679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T06:45:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2854752403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T06:45:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/992119679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T06:45:19.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2854752403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T06:45:19.864 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757275 got 163208757279 for osd.5 2024-02-10T06:45:19.864 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:20.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/992119679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T06:45:20.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2854752403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T06:45:20.931 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:20 smithi174 ceph-mon[29878]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:20.932 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:20 smithi174 ceph-mon[39523]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:21.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:20 smithi184 ceph-mon[33513]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:22.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018915 for osd.4 2024-02-10T06:45:22.156 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:22.210 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.844+0000 7f1d78d99700 1 -- 172.21.15.174:0/1940145935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d741015c0 msgr2=0x7f1d7410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.844+0000 7f1d78d99700 1 --2- 172.21.15.174:0/1940145935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d741015c0 0x7f1d7410daa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009f90 tx=0x7f1d6c00b330 comp rx=0 tx=0).stop 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 -- 172.21.15.174:0/1940145935 shutdown_connections 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 --2- 172.21.15.174:0/1940145935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d741015c0 0x7f1d7410daa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 --2- 172.21.15.174:0/1940145935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 0x7f1d74107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 --2- 172.21.15.174:0/1940145935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d74100c20 0x7f1d74101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 -- 172.21.15.174:0/1940145935 >> 172.21.15.174:0/1940145935 conn(0x7f1d74077b50 msgr2=0x7f1d74077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 -- 172.21.15.174:0/1940145935 shutdown_connections 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 -- 172.21.15.174:0/1940145935 wait complete. 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 Processor -- start 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 -- start start 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.845+0000 7f1d78d99700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d74100c20 0x7f1d741966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d741015c0 0x7f1d74196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:22.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 0x7f1d74197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a5db0 con 0x7f1d74100c20 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a5ef0 con 0x7f1d74106c60 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a60b0 con 0x7f1d741015c0 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d71d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d741015c0 0x7f1d74196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d74100c20 0x7f1d741966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d71d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d741015c0 0x7f1d74196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:42614/0 (socket says 172.21.15.174:42614) 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d71d9b700 1 -- 172.21.15.174:0/2167670963 learned_addr learned my addr 172.21.15.174:0/2167670963 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d72d9d700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 0x7f1d74197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 -- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d741015c0 msgr2=0x7f1d74196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d741015c0 0x7f1d74196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.212 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 -- 172.21.15.174:0/2167670963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 msgr2=0x7f1d74197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stdout:231928234004 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 0x7f1d74197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 -- 172.21.15.174:0/2167670963 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680095f0 con 0x7f1d74100c20 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d72d9d700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 0x7f1d74197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d7259c700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d74100c20 0x7f1d741966a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400d950 tx=0x7f1d6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64009930 con 0x7f1d74100c20 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c009c40 con 0x7f1d74100c20 2024-02-10T06:45:22.213 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1d64010460 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d741a6750 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.846+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400b5d0 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.848+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54005120 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.848+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f1d64009a90 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.849+0000 7f1d637fe700 1 --2- 172.21.15.174:0/2167670963 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1d5c06da80 0x7f1d5c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.849+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f1d6408cd30 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.849+0000 7f1d71d9b700 1 --2- 172.21.15.174:0/2167670963 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1d5c06da80 0x7f1d5c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.849+0000 7f1d71d9b700 1 --2- 172.21.15.174:0/2167670963 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1d5c06da80 0x7f1d5c06ff40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1d74198800 tx=0x7f1d68009310 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:21.853+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d6405a1c0 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.062+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1d540056a0 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.062+0000 7f1d637fe700 1 -- 172.21.15.174:0/2167670963 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d64059d50 con 0x7f1d74100c20 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1d5c06da80 msgr2=0x7f1d5c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:22.214 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 --2- 172.21.15.174:0/2167670963 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1d5c06da80 0x7f1d5c06ff40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1d74198800 tx=0x7f1d68009310 comp rx=0 tx=0).stop 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d74100c20 msgr2=0x7f1d741966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d74100c20 0x7f1d741966a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400d950 tx=0x7f1d6400dc60 comp rx=0 tx=0).stop 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 shutdown_connections 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 --2- 172.21.15.174:0/2167670963 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1d5c06da80 0x7f1d5c06ff40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d74106c60 0x7f1d74197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d74100c20 0x7f1d741966a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 --2- 172.21.15.174:0/2167670963 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d741015c0 0x7f1d74196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 >> 172.21.15.174:0/2167670963 conn(0x7f1d74077b50 msgr2=0x7f1d740ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 shutdown_connections 2024-02-10T06:45:22.215 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:22.064+0000 7f1d78d99700 1 -- 172.21.15.174:0/2167670963 wait complete. 2024-02-10T06:45:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:22 smithi174 ceph-mon[29878]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2167670963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T06:45:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:22 smithi174 ceph-mon[39523]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:22.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2167670963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T06:45:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:22 smithi184 ceph-mon[33513]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2167670963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T06:45:23.694 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234001 got 231928234004 for osd.7 2024-02-10T06:45:23.694 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:24.160 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:23 smithi174 ceph-mon[29878]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:24.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:23 smithi174 ceph-mon[39523]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/4086366351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 msgr2=0x7fd4c810eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/4086366351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c810eb70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fc0 tx=0x7fd4c000b330 comp rx=0 tx=0).stop 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/4086366351 shutdown_connections 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/4086366351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c810eb70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/4086366351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8107d50 0x7fd4c8108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/4086366351 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8101d70 0x7fd4c81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/4086366351 >> 172.21.15.174:0/4086366351 conn(0x7fd4c8078e50 msgr2=0x7fd4c8079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:24.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/4086366351 shutdown_connections 2024-02-10T06:45:24.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/4086366351 wait complete. 2024-02-10T06:45:24.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 Processor -- start 2024-02-10T06:45:24.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.270+0000 7fd4ce4d4700 1 -- start start 2024-02-10T06:45:24.273 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4ce4d4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8101d70 0x7fd4c8074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4ce4d4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c8074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4ce4d4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8107d50 0x7fd4c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4ce4d4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c81a5f10 con 0x7fd4c8102730 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4ce4d4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c81a6080 con 0x7fd4c8101d70 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4ce4d4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c81a6240 con 0x7fd4c8107d50 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c8074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c8074560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:60050/0 (socket says 172.21.15.174:60050) 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4cca71700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8107d50 0x7fd4c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 -- 172.21.15.174:0/2744516128 learned_addr learned my addr 172.21.15.174:0/2744516128 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c7fff700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8101d70 0x7fd4c8074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 -- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8107d50 msgr2=0x7fd4c8074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8107d50 0x7fd4c8074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 -- 172.21.15.174:0/2744516128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8101d70 msgr2=0x7fd4c8074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:24.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8101d70 0x7fd4c8074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 -- 172.21.15.174:0/2744516128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0095f0 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4cca71700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8107d50 0x7fd4c8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.271+0000 7fd4c77fe700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c8074560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00ec10 tx=0x7fd4bc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.272+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00cc90 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.272+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd4bc0044d0 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.272+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc010430 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.272+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c0009c40 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.272+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c81a6840 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.273+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c810a230 con 0x7fd4c8102730 2024-02-10T06:45:24.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.273+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fd4bc003f40 con 0x7fd4c8102730 2024-02-10T06:45:24.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.274+0000 7fd4c57fa700 1 --2- 172.21.15.174:0/2744516128 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd4b006da80 0x7fd4b006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:24.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.274+0000 7fd4c7fff700 1 --2- 172.21.15.174:0/2744516128 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd4b006da80 0x7fd4b006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:24.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.274+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fd4bc014070 con 0x7fd4c8102730 2024-02-10T06:45:24.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.274+0000 7fd4c7fff700 1 --2- 172.21.15.174:0/2744516128 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd4b006da80 0x7fd4b006ff40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8007b60 tx=0x7fd4b80073d0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:24.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.278+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4bc05ac60 con 0x7fd4c8102730 2024-02-10T06:45:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:23 smithi184 ceph-mon[33513]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:24.483 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.482+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd4c8075550 con 0x7fd4c8102730 2024-02-10T06:45:24.483 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.482+0000 7fd4c57fa700 1 -- 172.21.15.174:0/2744516128 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd4bc05a7f0 con 0x7fd4c8102730 2024-02-10T06:45:24.483 INFO:teuthology.orchestra.run.smithi174.stdout:77309411374 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd4b006da80 msgr2=0x7fd4b006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/2744516128 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd4b006da80 0x7fd4b006ff40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8007b60 tx=0x7fd4b80073d0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 msgr2=0x7fd4c8074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c8074560 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00ec10 tx=0x7fd4bc00ef20 comp rx=0 tx=0).stop 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 shutdown_connections 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c8107d50 0x7fd4c8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/2744516128 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd4b006da80 0x7fd4b006ff40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4c8101d70 0x7fd4c8074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.486 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 --2- 172.21.15.174:0/2744516128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c8102730 0x7fd4c8074560 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:24.486 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.484+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 >> 172.21.15.174:0/2744516128 conn(0x7fd4c8078e50 msgr2=0x7fd4c8100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:24.486 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.485+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 shutdown_connections 2024-02-10T06:45:24.486 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:24.485+0000 7fd4ce4d4700 1 -- 172.21.15.174:0/2744516128 wait complete. 2024-02-10T06:45:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2744516128' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T06:45:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2744516128' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T06:45:25.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2744516128' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T06:45:26.036 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:25 smithi174 ceph-mon[29878]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:26.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:25 smithi174 ceph-mon[39523]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:26.069 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411374 for osd.2 2024-02-10T06:45:26.069 DEBUG:teuthology.parallel:result is None 2024-02-10T06:45:26.069 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-10T06:45:26.070 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T06:45:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:25 smithi184 ceph-mon[33513]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:26.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:45:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:45:26.979 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:27.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:26 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:26 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:27.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:26 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:28.150 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:27 smithi174 ceph-mon[29878]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:28.151 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:45:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:45:28.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:27 smithi174 ceph-mon[39523]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:27 smithi184 ceph-mon[33513]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:29.985 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:29 smithi174 ceph-mon[29878]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:29.986 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:29 smithi174 ceph-mon[39523]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 -- 172.21.15.174:0/80506376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c106df0 msgr2=0x7ff07c1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:30.095 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 --2- 172.21.15.174:0/80506376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c106df0 0x7ff07c1071d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009920 tx=0x7ff06c009c30 comp rx=0 tx=0).stop 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 -- 172.21.15.174:0/80506376 shutdown_connections 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 --2- 172.21.15.174:0/80506376 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c1010e0 0x7ff07c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 --2- 172.21.15.174:0/80506376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c106df0 0x7ff07c1071d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 --2- 172.21.15.174:0/80506376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c100720 0x7ff07c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 -- 172.21.15.174:0/80506376 >> 172.21.15.174:0/80506376 conn(0x7ff07c075f70 msgr2=0x7ff07c076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 -- 172.21.15.174:0/80506376 shutdown_connections 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.091+0000 7ff0815f4700 1 -- 172.21.15.174:0/80506376 wait complete. 2024-02-10T06:45:30.096 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.092+0000 7ff0815f4700 1 Processor -- start 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.092+0000 7ff0815f4700 1 -- start start 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 0x7ff07c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c1010e0 0x7ff07c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c106df0 0x7ff07c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c197f20 con 0x7ff07c1010e0 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c198060 con 0x7ff07c106df0 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c198220 con 0x7ff07c100720 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 0x7ff07c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c1010e0 0x7ff07c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 0x7ff07c19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:40908/0 (socket says 172.21.15.174:40908) 2024-02-10T06:45:30.097 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 -- 172.21.15.174:0/758846217 learned_addr learned my addr 172.21.15.174:0/758846217 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07b7fe700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c106df0 0x7ff07c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 -- 172.21.15.174:0/758846217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c106df0 msgr2=0x7ff07c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c106df0 0x7ff07c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 -- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c1010e0 msgr2=0x7ff07c19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c1010e0 0x7ff07c19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 -- 172.21.15.174:0/758846217 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074009d00 con 0x7ff07c100720 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07b7fe700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c106df0 0x7ff07c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07a7fc700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c1010e0 0x7ff07c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff07affd700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 0x7ff07c19d700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00fa50 tx=0x7ff06c00fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c01c070 con 0x7ff07c100720 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff06c00bb10 con 0x7ff07c100720 2024-02-10T06:45:30.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0048f0 con 0x7ff07c100720 2024-02-10T06:45:30.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c009520 con 0x7ff07c100720 2024-02-10T06:45:30.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.093+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c198720 con 0x7ff07c100720 2024-02-10T06:45:30.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.094+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07c0689e0 con 0x7ff07c100720 2024-02-10T06:45:30.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.095+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7ff06c020070 con 0x7ff07c100720 2024-02-10T06:45:30.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.095+0000 7ff063fff700 1 --2- 172.21.15.174:0/758846217 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff06406da80 0x7ff06406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:30.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.095+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7ff06c08e220 con 0x7ff07c100720 2024-02-10T06:45:30.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.095+0000 7ff07a7fc700 1 --2- 172.21.15.174:0/758846217 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff06406da80 0x7ff06406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:30.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.096+0000 7ff07a7fc700 1 --2- 172.21.15.174:0/758846217 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff06406da80 0x7ff06406ff40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff07c106b40 tx=0x7ff07000f040 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:30.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.100+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff06c05b630 con 0x7ff07c100720 2024-02-10T06:45:30.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.292+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 --> v2:172.21.15.174:6800/1252702393 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff07c111590 con 0x7ff06406da80 2024-02-10T06:45:30.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.304+0000 7ff063fff700 1 -- 172.21.15.174:0/758846217 <== mgr.14568 v2:172.21.15.174:6800/1252702393 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384567 (secure 0 0 0) 0x7ff07c111590 con 0x7ff06406da80 2024-02-10T06:45:30.310 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:45:30.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff06406da80 msgr2=0x7ff06406ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:30.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 --2- 172.21.15.174:0/758846217 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff06406da80 0x7ff06406ff40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff07c106b40 tx=0x7ff07000f040 comp rx=0 tx=0).stop 2024-02-10T06:45:30.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 msgr2=0x7ff07c19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:30.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 0x7ff07c19d700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00fa50 tx=0x7ff06c00fa80 comp rx=0 tx=0).stop 2024-02-10T06:45:30.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 shutdown_connections 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 --2- 172.21.15.174:0/758846217 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff06406da80 0x7ff06406ff40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff07c106df0 0x7ff07c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff07c100720 0x7ff07c19d700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 --2- 172.21.15.174:0/758846217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff07c1010e0 0x7ff07c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 >> 172.21.15.174:0/758846217 conn(0x7ff07c075f70 msgr2=0x7ff07c105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 shutdown_connections 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:30.308+0000 7ff0815f4700 1 -- 172.21.15.174:0/758846217 wait complete. 2024-02-10T06:45:30.314 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2024-02-10T06:45:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:29 smithi184 ceph-mon[33513]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:30.616 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 podman[77108]: 2024-02-10 06:45:30.396024494 +0000 UTC m=+0.224553697 container create b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 podman[77108]: 2024-02-10 06:45:30.773528348 +0000 UTC m=+0.602057519 container init b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 podman[77108]: 2024-02-10 06:45:30.779042961 +0000 UTC m=+0.607572123 container start b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.791Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.791Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.792Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.174 port=9094 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.793Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.816Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.816Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.819Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:30.819Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-10T06:45:30.874 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 bash[77108]: b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de 2024-02-10T06:45:31.158 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:30 smithi174 systemd[1]: Started Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:45:31.158 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:31.158 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-mon[29878]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:45:31.158 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:30 smithi174 ceph-mon[39523]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:45:31] ENGINE Bus STOPPING 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:45:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:45:31] ENGINE Bus STOPPED 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:45:31] ENGINE Bus STARTING 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:45:31] ENGINE Serving on http://:::9283 2024-02-10T06:45:31.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:45:31] ENGINE Bus STARTED 2024-02-10T06:45:31.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:30 smithi184 ceph-mon[33513]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:45:31.547 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-02-10T06:45:28.888031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1011,"num_read_kb":857,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244240,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716976,"statfs":{"total":767960285184,"available":767710183424,"internally_reserved":0,"allocated":7467008,"data_stored":3991308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002831"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868368+0000","last_change":"2024-02-10T06:44:09.206711+0000","last_active":"2024-02-10T06:44:44.868368+0000","last_peered":"2024-02-10T06:44:44.868368+0000","last_clean":"2024-02-10T06:44:44.868368+0000","last_became_active":"2024-02-10T06:44:09.206417+0000","last_became_peered":"2024-02-10T06:44:09.206417+0000","last_unstale":"2024-02-10T06:44:44.868368+0000","last_undegraded":"2024-02-10T06:44:44.868368+0000","last_fullsized":"2024-02-10T06:44:44.868368+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:54:03.472311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493645+0000","last_change":"2024-02-10T06:44:07.206388+0000","last_active":"2024-02-10T06:44:45.493645+0000","last_peered":"2024-02-10T06:44:45.493645+0000","last_clean":"2024-02-10T06:44:45.493645+0000","last_became_active":"2024-02-10T06:44:07.206014+0000","last_became_peered":"2024-02-10T06:44:07.206014+0000","last_unstale":"2024-02-10T06:44:45.493645+0000","last_undegraded":"2024-02-10T06:44:45.493645+0000","last_fullsized":"2024-02-10T06:44:45.493645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:50:03.440192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868007+0000","last_change":"2024-02-10T06:44:03.133327+0000","last_active":"2024-02-10T06:44:44.868007+0000","last_peered":"2024-02-10T06:44:44.868007+0000","last_clean":"2024-02-10T06:44:44.868007+0000","last_became_active":"2024-02-10T06:44:03.132905+0000","last_became_peered":"2024-02-10T06:44:03.132905+0000","last_unstale":"2024-02-10T06:44:44.868007+0000","last_undegraded":"2024-02-10T06:44:44.868007+0000","last_fullsized":"2024-02-10T06:44:44.868007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:10:20.618882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868162+0000","last_change":"2024-02-10T06:44:05.190362+0000","last_active":"2024-02-10T06:44:44.868162+0000","last_peered":"2024-02-10T06:44:44.868162+0000","last_clean":"2024-02-10T06:44:44.868162+0000","last_became_active":"2024-02-10T06:44:05.190049+0000","last_became_peered":"2024-02-10T06:44:05.190049+0000","last_unstale":"2024-02-10T06:44:44.868162+0000","last_undegraded":"2024-02-10T06:44:44.868162+0000","last_fullsized":"2024-02-10T06:44:44.868162+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:37.660919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494258+0000","last_change":"2024-02-10T06:44:09.215857+0000","last_active":"2024-02-10T06:44:45.494258+0000","last_peered":"2024-02-10T06:44:45.494258+0000","last_clean":"2024-02-10T06:44:45.494258+0000","last_became_active":"2024-02-10T06:44:09.215591+0000","last_became_peered":"2024-02-10T06:44:09.215591+0000","last_unstale":"2024-02-10T06:44:45.494258+0000","last_undegraded":"2024-02-10T06:44:45.494258+0000","last_fullsized":"2024-02-10T06:44:45.494258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:11.588475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983135+0000","last_change":"2024-02-10T06:44:07.204319+0000","last_active":"2024-02-10T06:44:44.983135+0000","last_peered":"2024-02-10T06:44:44.983135+0000","last_clean":"2024-02-10T06:44:44.983135+0000","last_became_active":"2024-02-10T06:44:07.203958+0000","last_became_peered":"2024-02-10T06:44:07.203958+0000","last_unstale":"2024-02-10T06:44:44.983135+0000","last_undegraded":"2024-02-10T06:44:44.983135+0000","last_fullsized":"2024-02-10T06:44:44.983135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:09:20.748280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869181+0000","last_change":"2024-02-10T06:44:03.139168+0000","last_active":"2024-02-10T06:44:44.869181+0000","last_peered":"2024-02-10T06:44:44.869181+0000","last_clean":"2024-02-10T06:44:44.869181+0000","last_became_active":"2024-02-10T06:44:03.138865+0000","last_became_peered":"2024-02-10T06:44:03.138865+0000","last_unstale":"2024-02-10T06:44:44.869181+0000","last_undegraded":"2024-02-10T06:44:44.869181+0000","last_fullsized":"2024-02-10T06:44:44.869181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:20.211961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493803+0000","last_change":"2024-02-10T06:44:23.699705+0000","last_active":"2024-02-10T06:44:45.493803+0000","last_peered":"2024-02-10T06:44:45.493803+0000","last_clean":"2024-02-10T06:44:45.493803+0000","last_became_active":"2024-02-10T06:44:23.695308+0000","last_became_peered":"2024-02-10T06:44:23.695308+0000","last_unstale":"2024-02-10T06:44:45.493803+0000","last_undegraded":"2024-02-10T06:44:45.493803+0000","last_fullsized":"2024-02-10T06:44:45.493803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:49:21.319129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869856+0000","last_change":"2024-02-10T06:44:09.207726+0000","last_active":"2024-02-10T06:44:44.869856+0000","last_peered":"2024-02-10T06:44:44.869856+0000","last_clean":"2024-02-10T06:44:44.869856+0000","last_became_active":"2024-02-10T06:44:09.207436+0000","last_became_peered":"2024-02-10T06:44:09.207436+0000","last_unstale":"2024-02-10T06:44:44.869856+0000","last_undegraded":"2024-02-10T06:44:44.869856+0000","last_fullsized":"2024-02-10T06:44:44.869856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:10:17.572855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494367+0000","last_change":"2024-02-10T06:44:07.205472+0000","last_active":"2024-02-10T06:44:45.494367+0000","last_peered":"2024-02-10T06:44:45.494367+0000","last_clean":"2024-02-10T06:44:45.494367+0000","last_became_active":"2024-02-10T06:44:07.205112+0000","last_became_peered":"2024-02-10T06:44:07.205112+0000","last_unstale":"2024-02-10T06:44:45.494367+0000","last_undegraded":"2024-02-10T06:44:45.494367+0000","last_fullsized":"2024-02-10T06:44:45.494367+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:07.273157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919017+0000","last_change":"2024-02-10T06:44:23.671178+0000","last_active":"2024-02-10T06:44:44.919017+0000","last_peered":"2024-02-10T06:44:44.919017+0000","last_clean":"2024-02-10T06:44:44.919017+0000","last_became_active":"2024-02-10T06:44:23.670811+0000","last_became_peered":"2024-02-10T06:44:23.670811+0000","last_unstale":"2024-02-10T06:44:44.919017+0000","last_undegraded":"2024-02-10T06:44:44.919017+0000","last_fullsized":"2024-02-10T06:44:44.919017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:36.914773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493802+0000","last_change":"2024-02-10T06:44:23.604404+0000","last_active":"2024-02-10T06:44:45.493802+0000","last_peered":"2024-02-10T06:44:45.493802+0000","last_clean":"2024-02-10T06:44:45.493802+0000","last_became_active":"2024-02-10T06:44:23.600477+0000","last_became_peered":"2024-02-10T06:44:23.600477+0000","last_unstale":"2024-02-10T06:44:45.493802+0000","last_undegraded":"2024-02-10T06:44:45.493802+0000","last_fullsized":"2024-02-10T06:44:45.493802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:49:16.379676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867220+0000","last_change":"2024-02-10T06:44:09.207023+0000","last_active":"2024-02-10T06:44:44.867220+0000","last_peered":"2024-02-10T06:44:44.867220+0000","last_clean":"2024-02-10T06:44:44.867220+0000","last_became_active":"2024-02-10T06:44:09.206645+0000","last_became_peered":"2024-02-10T06:44:09.206645+0000","last_unstale":"2024-02-10T06:44:44.867220+0000","last_undegraded":"2024-02-10T06:44:44.867220+0000","last_fullsized":"2024-02-10T06:44:44.867220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:21:36.224394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870304+0000","last_change":"2024-02-10T06:44:07.204693+0000","last_active":"2024-02-10T06:44:44.870304+0000","last_peered":"2024-02-10T06:44:44.870304+0000","last_clean":"2024-02-10T06:44:44.870304+0000","last_became_active":"2024-02-10T06:44:07.204377+0000","last_became_peered":"2024-02-10T06:44:07.204377+0000","last_unstale":"2024-02-10T06:44:44.870304+0000","last_undegraded":"2024-02-10T06:44:44.870304+0000","last_fullsized":"2024-02-10T06:44:44.870304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:09.733042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495181+0000","last_change":"2024-02-10T06:44:03.135904+0000","last_active":"2024-02-10T06:44:45.495181+0000","last_peered":"2024-02-10T06:44:45.495181+0000","last_clean":"2024-02-10T06:44:45.495181+0000","last_became_active":"2024-02-10T06:44:03.135624+0000","last_became_peered":"2024-02-10T06:44:03.135624+0000","last_unstale":"2024-02-10T06:44:45.495181+0000","last_undegraded":"2024-02-10T06:44:45.495181+0000","last_fullsized":"2024-02-10T06:44:45.495181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:34:24.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493640+0000","last_change":"2024-02-10T06:44:05.190339+0000","last_active":"2024-02-10T06:44:45.493640+0000","last_peered":"2024-02-10T06:44:45.493640+0000","last_clean":"2024-02-10T06:44:45.493640+0000","last_became_active":"2024-02-10T06:44:05.190029+0000","last_became_peered":"2024-02-10T06:44:05.190029+0000","last_unstale":"2024-02-10T06:44:45.493640+0000","last_undegraded":"2024-02-10T06:44:45.493640+0000","last_fullsized":"2024-02-10T06:44:45.493640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:19:24.957844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869422+0000","last_change":"2024-02-10T06:44:09.207570+0000","last_active":"2024-02-10T06:44:44.869422+0000","last_peered":"2024-02-10T06:44:44.869422+0000","last_clean":"2024-02-10T06:44:44.869422+0000","last_became_active":"2024-02-10T06:44:09.207300+0000","last_became_peered":"2024-02-10T06:44:09.207300+0000","last_unstale":"2024-02-10T06:44:44.869422+0000","last_undegraded":"2024-02-10T06:44:44.869422+0000","last_fullsized":"2024-02-10T06:44:44.869422+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:00:02.470714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493968+0000","last_change":"2024-02-10T06:44:07.193765+0000","last_active":"2024-02-10T06:44:45.493968+0000","last_peered":"2024-02-10T06:44:45.493968+0000","last_clean":"2024-02-10T06:44:45.493968+0000","last_became_active":"2024-02-10T06:44:07.193462+0000","last_became_peered":"2024-02-10T06:44:07.193462+0000","last_unstale":"2024-02-10T06:44:45.493968+0000","last_undegraded":"2024-02-10T06:44:45.493968+0000","last_fullsized":"2024-02-10T06:44:45.493968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:06:52.310075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868613+0000","last_change":"2024-02-10T06:44:03.133714+0000","last_active":"2024-02-10T06:44:44.868613+0000","last_peered":"2024-02-10T06:44:44.868613+0000","last_clean":"2024-02-10T06:44:44.868613+0000","last_became_active":"2024-02-10T06:44:03.133395+0000","last_became_peered":"2024-02-10T06:44:03.133395+0000","last_unstale":"2024-02-10T06:44:44.868613+0000","last_undegraded":"2024-02-10T06:44:44.868613+0000","last_fullsized":"2024-02-10T06:44:44.868613+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:18:18.057639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868947+0000","last_change":"2024-02-10T06:44:05.198626+0000","last_active":"2024-02-10T06:44:44.868947+0000","last_peered":"2024-02-10T06:44:44.868947+0000","last_clean":"2024-02-10T06:44:44.868947+0000","last_became_active":"2024-02-10T06:44:05.198039+0000","last_became_peered":"2024-02-10T06:44:05.198039+0000","last_unstale":"2024-02-10T06:44:44.868947+0000","last_undegraded":"2024-02-10T06:44:44.868947+0000","last_fullsized":"2024-02-10T06:44:44.868947+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:52:02.512152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495646+0000","last_change":"2024-02-10T06:44:09.210492+0000","last_active":"2024-02-10T06:44:45.495646+0000","last_peered":"2024-02-10T06:44:45.495646+0000","last_clean":"2024-02-10T06:44:45.495646+0000","last_became_active":"2024-02-10T06:44:09.210162+0000","last_became_peered":"2024-02-10T06:44:09.210162+0000","last_unstale":"2024-02-10T06:44:45.495646+0000","last_undegraded":"2024-02-10T06:44:45.495646+0000","last_fullsized":"2024-02-10T06:44:45.495646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:02:02.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984684+0000","last_change":"2024-02-10T06:44:07.206349+0000","last_active":"2024-02-10T06:44:44.984684+0000","last_peered":"2024-02-10T06:44:44.984684+0000","last_clean":"2024-02-10T06:44:44.984684+0000","last_became_active":"2024-02-10T06:44:07.205996+0000","last_became_peered":"2024-02-10T06:44:07.205996+0000","last_unstale":"2024-02-10T06:44:44.984684+0000","last_undegraded":"2024-02-10T06:44:44.984684+0000","last_fullsized":"2024-02-10T06:44:44.984684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:40:27.716618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496018+0000","last_change":"2024-02-10T06:44:03.133745+0000","last_active":"2024-02-10T06:44:45.496018+0000","last_peered":"2024-02-10T06:44:45.496018+0000","last_clean":"2024-02-10T06:44:45.496018+0000","last_became_active":"2024-02-10T06:44:03.133024+0000","last_became_peered":"2024-02-10T06:44:03.133024+0000","last_unstale":"2024-02-10T06:44:45.496018+0000","last_undegraded":"2024-02-10T06:44:45.496018+0000","last_fullsized":"2024-02-10T06:44:45.496018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:50:02.399559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919354+0000","last_change":"2024-02-10T06:44:05.196604+0000","last_active":"2024-02-10T06:44:44.919354+0000","last_peered":"2024-02-10T06:44:44.919354+0000","last_clean":"2024-02-10T06:44:44.919354+0000","last_became_active":"2024-02-10T06:44:05.196319+0000","last_became_peered":"2024-02-10T06:44:05.196319+0000","last_unstale":"2024-02-10T06:44:44.919354+0000","last_undegraded":"2024-02-10T06:44:44.919354+0000","last_fullsized":"2024-02-10T06:44:44.919354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:20:40.973169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984163+0000","last_change":"2024-02-10T06:44:09.212123+0000","last_active":"2024-02-10T06:44:44.984163+0000","last_peered":"2024-02-10T06:44:44.984163+0000","last_clean":"2024-02-10T06:44:44.984163+0000","last_became_active":"2024-02-10T06:44:09.211826+0000","last_became_peered":"2024-02-10T06:44:09.211826+0000","last_unstale":"2024-02-10T06:44:44.984163+0000","last_undegraded":"2024-02-10T06:44:44.984163+0000","last_fullsized":"2024-02-10T06:44:44.984163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:11:48.697285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868702+0000","last_change":"2024-02-10T06:44:07.197269+0000","last_active":"2024-02-10T06:44:44.868702+0000","last_peered":"2024-02-10T06:44:44.868702+0000","last_clean":"2024-02-10T06:44:44.868702+0000","last_became_active":"2024-02-10T06:44:07.196949+0000","last_became_peered":"2024-02-10T06:44:07.196949+0000","last_unstale":"2024-02-10T06:44:44.868702+0000","last_undegraded":"2024-02-10T06:44:44.868702+0000","last_fullsized":"2024-02-10T06:44:44.868702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:16:51.826635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"60'1","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868298+0000","last_change":"2024-02-10T06:44:03.134052+0000","last_active":"2024-02-10T06:44:44.868298+0000","last_peered":"2024-02-10T06:44:44.868298+0000","last_clean":"2024-02-10T06:44:44.868298+0000","last_became_active":"2024-02-10T06:44:03.133505+0000","last_became_peered":"2024-02-10T06:44:03.133505+0000","last_unstale":"2024-02-10T06:44:44.868298+0000","last_undegraded":"2024-02-10T06:44:44.868298+0000","last_fullsized":"2024-02-10T06:44:44.868298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:09.097584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496171+0000","last_change":"2024-02-10T06:44:05.197133+0000","last_active":"2024-02-10T06:44:45.496171+0000","last_peered":"2024-02-10T06:44:45.496171+0000","last_clean":"2024-02-10T06:44:45.496171+0000","last_became_active":"2024-02-10T06:44:05.196816+0000","last_became_peered":"2024-02-10T06:44:05.196816+0000","last_unstale":"2024-02-10T06:44:45.496171+0000","last_undegraded":"2024-02-10T06:44:45.496171+0000","last_fullsized":"2024-02-10T06:44:45.496171+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:53:30.745628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495515+0000","last_change":"2024-02-10T06:44:09.214454+0000","last_active":"2024-02-10T06:44:45.495515+0000","last_peered":"2024-02-10T06:44:45.495515+0000","last_clean":"2024-02-10T06:44:45.495515+0000","last_became_active":"2024-02-10T06:44:09.214129+0000","last_became_peered":"2024-02-10T06:44:09.214129+0000","last_unstale":"2024-02-10T06:44:45.495515+0000","last_undegraded":"2024-02-10T06:44:45.495515+0000","last_fullsized":"2024-02-10T06:44:45.495515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:03:27.408367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193730+0000","last_change":"2024-02-10T06:44:07.201769+0000","last_active":"2024-02-10T06:45:18.193730+0000","last_peered":"2024-02-10T06:45:18.193730+0000","last_clean":"2024-02-10T06:45:18.193730+0000","last_became_active":"2024-02-10T06:44:07.201482+0000","last_became_peered":"2024-02-10T06:44:07.201482+0000","last_unstale":"2024-02-10T06:45:18.193730+0000","last_undegraded":"2024-02-10T06:45:18.193730+0000","last_fullsized":"2024-02-10T06:45:18.193730+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:14:39.019619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869815+0000","last_change":"2024-02-10T06:44:03.130230+0000","last_active":"2024-02-10T06:44:44.869815+0000","last_peered":"2024-02-10T06:44:44.869815+0000","last_clean":"2024-02-10T06:44:44.869815+0000","last_became_active":"2024-02-10T06:44:03.129949+0000","last_became_peered":"2024-02-10T06:44:03.129949+0000","last_unstale":"2024-02-10T06:44:44.869815+0000","last_undegraded":"2024-02-10T06:44:44.869815+0000","last_fullsized":"2024-02-10T06:44:44.869815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:00:31.285542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868583+0000","last_change":"2024-02-10T06:44:05.332344+0000","last_active":"2024-02-10T06:44:44.868583+0000","last_peered":"2024-02-10T06:44:44.868583+0000","last_clean":"2024-02-10T06:44:44.868583+0000","last_became_active":"2024-02-10T06:44:05.332099+0000","last_became_peered":"2024-02-10T06:44:05.332099+0000","last_unstale":"2024-02-10T06:44:44.868583+0000","last_undegraded":"2024-02-10T06:44:44.868583+0000","last_fullsized":"2024-02-10T06:44:44.868583+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:34:11.834836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868887+0000","last_change":"2024-02-10T06:44:09.201694+0000","last_active":"2024-02-10T06:44:44.868887+0000","last_peered":"2024-02-10T06:44:44.868887+0000","last_clean":"2024-02-10T06:44:44.868887+0000","last_became_active":"2024-02-10T06:44:09.201290+0000","last_became_peered":"2024-02-10T06:44:09.201290+0000","last_unstale":"2024-02-10T06:44:44.868887+0000","last_undegraded":"2024-02-10T06:44:44.868887+0000","last_fullsized":"2024-02-10T06:44:44.868887+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:46:55.440561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494601+0000","last_change":"2024-02-10T06:44:07.203652+0000","last_active":"2024-02-10T06:44:45.494601+0000","last_peered":"2024-02-10T06:44:45.494601+0000","last_clean":"2024-02-10T06:44:45.494601+0000","last_became_active":"2024-02-10T06:44:07.203355+0000","last_became_peered":"2024-02-10T06:44:07.203355+0000","last_unstale":"2024-02-10T06:44:45.494601+0000","last_undegraded":"2024-02-10T06:44:45.494601+0000","last_fullsized":"2024-02-10T06:44:45.494601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:30:15.283761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496572+0000","last_change":"2024-02-10T06:44:03.131244+0000","last_active":"2024-02-10T06:44:45.496572+0000","last_peered":"2024-02-10T06:44:45.496572+0000","last_clean":"2024-02-10T06:44:45.496572+0000","last_became_active":"2024-02-10T06:44:03.130884+0000","last_became_peered":"2024-02-10T06:44:03.130884+0000","last_unstale":"2024-02-10T06:44:45.496572+0000","last_undegraded":"2024-02-10T06:44:45.496572+0000","last_fullsized":"2024-02-10T06:44:45.496572+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:42:21.848411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868633+0000","last_change":"2024-02-10T06:44:05.332063+0000","last_active":"2024-02-10T06:44:44.868633+0000","last_peered":"2024-02-10T06:44:44.868633+0000","last_clean":"2024-02-10T06:44:44.868633+0000","last_became_active":"2024-02-10T06:44:05.331667+0000","last_became_peered":"2024-02-10T06:44:05.331667+0000","last_unstale":"2024-02-10T06:44:44.868633+0000","last_undegraded":"2024-02-10T06:44:44.868633+0000","last_fullsized":"2024-02-10T06:44:44.868633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:44:09.459671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494138+0000","last_change":"2024-02-10T06:44:09.213087+0000","last_active":"2024-02-10T06:44:45.494138+0000","last_peered":"2024-02-10T06:44:45.494138+0000","last_clean":"2024-02-10T06:44:45.494138+0000","last_became_active":"2024-02-10T06:44:09.212681+0000","last_became_peered":"2024-02-10T06:44:09.212681+0000","last_unstale":"2024-02-10T06:44:45.494138+0000","last_undegraded":"2024-02-10T06:44:45.494138+0000","last_fullsized":"2024-02-10T06:44:45.494138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:57:50.506314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494972+0000","last_change":"2024-02-10T06:44:07.200774+0000","last_active":"2024-02-10T06:44:45.494972+0000","last_peered":"2024-02-10T06:44:45.494972+0000","last_clean":"2024-02-10T06:44:45.494972+0000","last_became_active":"2024-02-10T06:44:07.200506+0000","last_became_peered":"2024-02-10T06:44:07.200506+0000","last_unstale":"2024-02-10T06:44:45.494972+0000","last_undegraded":"2024-02-10T06:44:45.494972+0000","last_fullsized":"2024-02-10T06:44:45.494972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:07:59.365259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870817+0000","last_change":"2024-02-10T06:44:03.136894+0000","last_active":"2024-02-10T06:44:44.870817+0000","last_peered":"2024-02-10T06:44:44.870817+0000","last_clean":"2024-02-10T06:44:44.870817+0000","last_became_active":"2024-02-10T06:44:03.136599+0000","last_became_peered":"2024-02-10T06:44:03.136599+0000","last_unstale":"2024-02-10T06:44:44.870817+0000","last_undegraded":"2024-02-10T06:44:44.870817+0000","last_fullsized":"2024-02-10T06:44:44.870817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:53:00.098332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868025+0000","last_change":"2024-02-10T06:44:05.197443+0000","last_active":"2024-02-10T06:44:44.868025+0000","last_peered":"2024-02-10T06:44:44.868025+0000","last_clean":"2024-02-10T06:44:44.868025+0000","last_became_active":"2024-02-10T06:44:05.197025+0000","last_became_peered":"2024-02-10T06:44:05.197025+0000","last_unstale":"2024-02-10T06:44:44.868025+0000","last_undegraded":"2024-02-10T06:44:44.868025+0000","last_fullsized":"2024-02-10T06:44:44.868025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:46:36.237275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868051+0000","last_change":"2024-02-10T06:44:09.205073+0000","last_active":"2024-02-10T06:44:44.868051+0000","last_peered":"2024-02-10T06:44:44.868051+0000","last_clean":"2024-02-10T06:44:44.868051+0000","last_became_active":"2024-02-10T06:44:09.204758+0000","last_became_peered":"2024-02-10T06:44:09.204758+0000","last_unstale":"2024-02-10T06:44:44.868051+0000","last_undegraded":"2024-02-10T06:44:44.868051+0000","last_fullsized":"2024-02-10T06:44:44.868051+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:04:18.216542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983702+0000","last_change":"2024-02-10T06:44:07.203090+0000","last_active":"2024-02-10T06:44:44.983702+0000","last_peered":"2024-02-10T06:44:44.983702+0000","last_clean":"2024-02-10T06:44:44.983702+0000","last_became_active":"2024-02-10T06:44:07.202791+0000","last_became_peered":"2024-02-10T06:44:07.202791+0000","last_unstale":"2024-02-10T06:44:44.983702+0000","last_undegraded":"2024-02-10T06:44:44.983702+0000","last_fullsized":"2024-02-10T06:44:44.983702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:37.600065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984329+0000","last_change":"2024-02-10T06:44:03.139011+0000","last_active":"2024-02-10T06:44:44.984329+0000","last_peered":"2024-02-10T06:44:44.984329+0000","last_clean":"2024-02-10T06:44:44.984329+0000","last_became_active":"2024-02-10T06:44:03.138713+0000","last_became_peered":"2024-02-10T06:44:03.138713+0000","last_unstale":"2024-02-10T06:44:44.984329+0000","last_undegraded":"2024-02-10T06:44:44.984329+0000","last_fullsized":"2024-02-10T06:44:44.984329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:53.842077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493374+0000","last_change":"2024-02-10T06:44:05.198245+0000","last_active":"2024-02-10T06:44:45.493374+0000","last_peered":"2024-02-10T06:44:45.493374+0000","last_clean":"2024-02-10T06:44:45.493374+0000","last_became_active":"2024-02-10T06:44:05.197240+0000","last_became_peered":"2024-02-10T06:44:05.197240+0000","last_unstale":"2024-02-10T06:44:45.493374+0000","last_undegraded":"2024-02-10T06:44:45.493374+0000","last_fullsized":"2024-02-10T06:44:45.493374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:06:41.694172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.866813+0000","last_change":"2024-02-10T06:44:09.205396+0000","last_active":"2024-02-10T06:44:44.866813+0000","last_peered":"2024-02-10T06:44:44.866813+0000","last_clean":"2024-02-10T06:44:44.866813+0000","last_became_active":"2024-02-10T06:44:09.205124+0000","last_became_peered":"2024-02-10T06:44:09.205124+0000","last_unstale":"2024-02-10T06:44:44.866813+0000","last_undegraded":"2024-02-10T06:44:44.866813+0000","last_fullsized":"2024-02-10T06:44:44.866813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:12:27.596628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868701+0000","last_change":"2024-02-10T06:44:07.197256+0000","last_active":"2024-02-10T06:44:44.868701+0000","last_peered":"2024-02-10T06:44:44.868701+0000","last_clean":"2024-02-10T06:44:44.868701+0000","last_became_active":"2024-02-10T06:44:07.196972+0000","last_became_peered":"2024-02-10T06:44:07.196972+0000","last_unstale":"2024-02-10T06:44:44.868701+0000","last_undegraded":"2024-02-10T06:44:44.868701+0000","last_fullsized":"2024-02-10T06:44:44.868701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:08:02.938962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496327+0000","last_change":"2024-02-10T06:44:03.131992+0000","last_active":"2024-02-10T06:44:45.496327+0000","last_peered":"2024-02-10T06:44:45.496327+0000","last_clean":"2024-02-10T06:44:45.496327+0000","last_became_active":"2024-02-10T06:44:03.131666+0000","last_became_peered":"2024-02-10T06:44:03.131666+0000","last_unstale":"2024-02-10T06:44:45.496327+0000","last_undegraded":"2024-02-10T06:44:45.496327+0000","last_fullsized":"2024-02-10T06:44:45.496327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:39:04.101751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983381+0000","last_change":"2024-02-10T06:44:05.195591+0000","last_active":"2024-02-10T06:44:44.983381+0000","last_peered":"2024-02-10T06:44:44.983381+0000","last_clean":"2024-02-10T06:44:44.983381+0000","last_became_active":"2024-02-10T06:44:05.195292+0000","last_became_peered":"2024-02-10T06:44:05.195292+0000","last_unstale":"2024-02-10T06:44:44.983381+0000","last_undegraded":"2024-02-10T06:44:44.983381+0000","last_fullsized":"2024-02-10T06:44:44.983381+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:45:21.870785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495485+0000","last_change":"2024-02-10T06:44:09.208960+0000","last_active":"2024-02-10T06:44:45.495485+0000","last_peered":"2024-02-10T06:44:45.495485+0000","last_clean":"2024-02-10T06:44:45.495485+0000","last_became_active":"2024-02-10T06:44:09.208668+0000","last_became_peered":"2024-02-10T06:44:09.208668+0000","last_unstale":"2024-02-10T06:44:45.495485+0000","last_undegraded":"2024-02-10T06:44:45.495485+0000","last_fullsized":"2024-02-10T06:44:45.495485+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:53:49.029054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193452+0000","last_change":"2024-02-10T06:44:07.199098+0000","last_active":"2024-02-10T06:45:18.193452+0000","last_peered":"2024-02-10T06:45:18.193452+0000","last_clean":"2024-02-10T06:45:18.193452+0000","last_became_active":"2024-02-10T06:44:07.198779+0000","last_became_peered":"2024-02-10T06:44:07.198779+0000","last_unstale":"2024-02-10T06:45:18.193452+0000","last_undegraded":"2024-02-10T06:45:18.193452+0000","last_fullsized":"2024-02-10T06:45:18.193452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:06:04.122479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"60'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867368+0000","last_change":"2024-02-10T06:44:03.139192+0000","last_active":"2024-02-10T06:44:44.867368+0000","last_peered":"2024-02-10T06:44:44.867368+0000","last_clean":"2024-02-10T06:44:44.867368+0000","last_became_active":"2024-02-10T06:44:03.138301+0000","last_became_peered":"2024-02-10T06:44:03.138301+0000","last_unstale":"2024-02-10T06:44:44.867368+0000","last_undegraded":"2024-02-10T06:44:44.867368+0000","last_fullsized":"2024-02-10T06:44:44.867368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:08:28.173573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870838+0000","last_change":"2024-02-10T06:44:05.332054+0000","last_active":"2024-02-10T06:44:44.870838+0000","last_peered":"2024-02-10T06:44:44.870838+0000","last_clean":"2024-02-10T06:44:44.870838+0000","last_became_active":"2024-02-10T06:44:05.331722+0000","last_became_peered":"2024-02-10T06:44:05.331722+0000","last_unstale":"2024-02-10T06:44:44.870838+0000","last_undegraded":"2024-02-10T06:44:44.870838+0000","last_fullsized":"2024-02-10T06:44:44.870838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:57:06.187835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868159+0000","last_change":"2024-02-10T06:44:09.206729+0000","last_active":"2024-02-10T06:44:44.868159+0000","last_peered":"2024-02-10T06:44:44.868159+0000","last_clean":"2024-02-10T06:44:44.868159+0000","last_became_active":"2024-02-10T06:44:09.206375+0000","last_became_peered":"2024-02-10T06:44:09.206375+0000","last_unstale":"2024-02-10T06:44:44.868159+0000","last_undegraded":"2024-02-10T06:44:44.868159+0000","last_fullsized":"2024-02-10T06:44:44.868159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:11:32.326758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193632+0000","last_change":"2024-02-10T06:44:07.201557+0000","last_active":"2024-02-10T06:45:18.193632+0000","last_peered":"2024-02-10T06:45:18.193632+0000","last_clean":"2024-02-10T06:45:18.193632+0000","last_became_active":"2024-02-10T06:44:07.201263+0000","last_became_peered":"2024-02-10T06:44:07.201263+0000","last_unstale":"2024-02-10T06:45:18.193632+0000","last_undegraded":"2024-02-10T06:45:18.193632+0000","last_fullsized":"2024-02-10T06:45:18.193632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:31:44.221235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869986+0000","last_change":"2024-02-10T06:44:03.131413+0000","last_active":"2024-02-10T06:44:44.869986+0000","last_peered":"2024-02-10T06:44:44.869986+0000","last_clean":"2024-02-10T06:44:44.869986+0000","last_became_active":"2024-02-10T06:44:03.131137+0000","last_became_peered":"2024-02-10T06:44:03.131137+0000","last_unstale":"2024-02-10T06:44:44.869986+0000","last_undegraded":"2024-02-10T06:44:44.869986+0000","last_fullsized":"2024-02-10T06:44:44.869986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:08:35.332808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868460+0000","last_change":"2024-02-10T06:44:05.332734+0000","last_active":"2024-02-10T06:44:44.868460+0000","last_peered":"2024-02-10T06:44:44.868460+0000","last_clean":"2024-02-10T06:44:44.868460+0000","last_became_active":"2024-02-10T06:44:05.332217+0000","last_became_peered":"2024-02-10T06:44:05.332217+0000","last_unstale":"2024-02-10T06:44:44.868460+0000","last_undegraded":"2024-02-10T06:44:44.868460+0000","last_fullsized":"2024-02-10T06:44:44.868460+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:15:07.935178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495713+0000","last_change":"2024-02-10T06:44:09.214695+0000","last_active":"2024-02-10T06:44:45.495713+0000","last_peered":"2024-02-10T06:44:45.495713+0000","last_clean":"2024-02-10T06:44:45.495713+0000","last_became_active":"2024-02-10T06:44:09.214422+0000","last_became_peered":"2024-02-10T06:44:09.214422+0000","last_unstale":"2024-02-10T06:44:45.495713+0000","last_undegraded":"2024-02-10T06:44:45.495713+0000","last_fullsized":"2024-02-10T06:44:45.495713+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:45:09.044749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870735+0000","last_change":"2024-02-10T06:44:07.206445+0000","last_active":"2024-02-10T06:44:44.870735+0000","last_peered":"2024-02-10T06:44:44.870735+0000","last_clean":"2024-02-10T06:44:44.870735+0000","last_became_active":"2024-02-10T06:44:07.206157+0000","last_became_peered":"2024-02-10T06:44:07.206157+0000","last_unstale":"2024-02-10T06:44:44.870735+0000","last_undegraded":"2024-02-10T06:44:44.870735+0000","last_fullsized":"2024-02-10T06:44:44.870735+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:12:44.835653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496485+0000","last_change":"2024-02-10T06:44:03.132818+0000","last_active":"2024-02-10T06:44:45.496485+0000","last_peered":"2024-02-10T06:44:45.496485+0000","last_clean":"2024-02-10T06:44:45.496485+0000","last_became_active":"2024-02-10T06:44:03.132358+0000","last_became_peered":"2024-02-10T06:44:03.132358+0000","last_unstale":"2024-02-10T06:44:45.496485+0000","last_undegraded":"2024-02-10T06:44:45.496485+0000","last_fullsized":"2024-02-10T06:44:45.496485+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:51:30.613464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869404+0000","last_change":"2024-02-10T06:44:05.331978+0000","last_active":"2024-02-10T06:44:44.869404+0000","last_peered":"2024-02-10T06:44:44.869404+0000","last_clean":"2024-02-10T06:44:44.869404+0000","last_became_active":"2024-02-10T06:44:05.331646+0000","last_became_peered":"2024-02-10T06:44:05.331646+0000","last_unstale":"2024-02-10T06:44:44.869404+0000","last_undegraded":"2024-02-10T06:44:44.869404+0000","last_fullsized":"2024-02-10T06:44:44.869404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:59:37.407166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919060+0000","last_change":"2024-02-10T06:44:09.211639+0000","last_active":"2024-02-10T06:44:44.919060+0000","last_peered":"2024-02-10T06:44:44.919060+0000","last_clean":"2024-02-10T06:44:44.919060+0000","last_became_active":"2024-02-10T06:44:09.211286+0000","last_became_peered":"2024-02-10T06:44:09.211286+0000","last_unstale":"2024-02-10T06:44:44.919060+0000","last_undegraded":"2024-02-10T06:44:44.919060+0000","last_fullsized":"2024-02-10T06:44:44.919060+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:09:41.661125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869509+0000","last_change":"2024-02-10T06:44:07.198398+0000","last_active":"2024-02-10T06:44:44.869509+0000","last_peered":"2024-02-10T06:44:44.869509+0000","last_clean":"2024-02-10T06:44:44.869509+0000","last_became_active":"2024-02-10T06:44:07.198073+0000","last_became_peered":"2024-02-10T06:44:07.198073+0000","last_unstale":"2024-02-10T06:44:44.869509+0000","last_undegraded":"2024-02-10T06:44:44.869509+0000","last_fullsized":"2024-02-10T06:44:44.869509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:19:16.672133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918812+0000","last_change":"2024-02-10T06:44:03.142957+0000","last_active":"2024-02-10T06:44:44.918812+0000","last_peered":"2024-02-10T06:44:44.918812+0000","last_clean":"2024-02-10T06:44:44.918812+0000","last_became_active":"2024-02-10T06:44:03.142675+0000","last_became_peered":"2024-02-10T06:44:03.142675+0000","last_unstale":"2024-02-10T06:44:44.918812+0000","last_undegraded":"2024-02-10T06:44:44.918812+0000","last_fullsized":"2024-02-10T06:44:44.918812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:06:47.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868355+0000","last_change":"2024-02-10T06:44:05.331521+0000","last_active":"2024-02-10T06:44:44.868355+0000","last_peered":"2024-02-10T06:44:44.868355+0000","last_clean":"2024-02-10T06:44:44.868355+0000","last_became_active":"2024-02-10T06:44:05.331221+0000","last_became_peered":"2024-02-10T06:44:05.331221+0000","last_unstale":"2024-02-10T06:44:44.868355+0000","last_undegraded":"2024-02-10T06:44:44.868355+0000","last_fullsized":"2024-02-10T06:44:44.868355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:02:31.669237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868788+0000","last_change":"2024-02-10T06:44:09.206739+0000","last_active":"2024-02-10T06:44:44.868788+0000","last_peered":"2024-02-10T06:44:44.868788+0000","last_clean":"2024-02-10T06:44:44.868788+0000","last_became_active":"2024-02-10T06:44:09.206429+0000","last_became_peered":"2024-02-10T06:44:09.206429+0000","last_unstale":"2024-02-10T06:44:44.868788+0000","last_undegraded":"2024-02-10T06:44:44.868788+0000","last_fullsized":"2024-02-10T06:44:44.868788+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:51:42.777986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495265+0000","last_change":"2024-02-10T06:44:03.137085+0000","last_active":"2024-02-10T06:44:45.495265+0000","last_peered":"2024-02-10T06:44:45.495265+0000","last_clean":"2024-02-10T06:44:45.495265+0000","last_became_active":"2024-02-10T06:44:03.136782+0000","last_became_peered":"2024-02-10T06:44:03.136782+0000","last_unstale":"2024-02-10T06:44:45.495265+0000","last_undegraded":"2024-02-10T06:44:45.495265+0000","last_fullsized":"2024-02-10T06:44:45.495265+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:19:33.094592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493134+0000","last_change":"2024-02-10T06:44:05.196500+0000","last_active":"2024-02-10T06:44:45.493134+0000","last_peered":"2024-02-10T06:44:45.493134+0000","last_clean":"2024-02-10T06:44:45.493134+0000","last_became_active":"2024-02-10T06:44:05.196234+0000","last_became_peered":"2024-02-10T06:44:05.196234+0000","last_unstale":"2024-02-10T06:44:45.493134+0000","last_undegraded":"2024-02-10T06:44:45.493134+0000","last_fullsized":"2024-02-10T06:44:45.493134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:17:33.731568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918808+0000","last_change":"2024-02-10T06:44:07.202717+0000","last_active":"2024-02-10T06:44:44.918808+0000","last_peered":"2024-02-10T06:44:44.918808+0000","last_clean":"2024-02-10T06:44:44.918808+0000","last_became_active":"2024-02-10T06:44:07.202403+0000","last_became_peered":"2024-02-10T06:44:07.202403+0000","last_unstale":"2024-02-10T06:44:44.918808+0000","last_undegraded":"2024-02-10T06:44:44.918808+0000","last_fullsized":"2024-02-10T06:44:44.918808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:45:16.320601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.866813+0000","last_change":"2024-02-10T06:44:09.200057+0000","last_active":"2024-02-10T06:44:44.866813+0000","last_peered":"2024-02-10T06:44:44.866813+0000","last_clean":"2024-02-10T06:44:44.866813+0000","last_became_active":"2024-02-10T06:44:09.199734+0000","last_became_peered":"2024-02-10T06:44:09.199734+0000","last_unstale":"2024-02-10T06:44:44.866813+0000","last_undegraded":"2024-02-10T06:44:44.866813+0000","last_fullsized":"2024-02-10T06:44:44.866813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:57:10.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984070+0000","last_change":"2024-02-10T06:44:03.139457+0000","last_active":"2024-02-10T06:44:44.984070+0000","last_peered":"2024-02-10T06:44:44.984070+0000","last_clean":"2024-02-10T06:44:44.984070+0000","last_became_active":"2024-02-10T06:44:03.139178+0000","last_became_peered":"2024-02-10T06:44:03.139178+0000","last_unstale":"2024-02-10T06:44:44.984070+0000","last_undegraded":"2024-02-10T06:44:44.984070+0000","last_fullsized":"2024-02-10T06:44:44.984070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:41:42.617087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983450+0000","last_change":"2024-02-10T06:44:05.199342+0000","last_active":"2024-02-10T06:44:44.983450+0000","last_peered":"2024-02-10T06:44:44.983450+0000","last_clean":"2024-02-10T06:44:44.983450+0000","last_became_active":"2024-02-10T06:44:05.198930+0000","last_became_peered":"2024-02-10T06:44:05.198930+0000","last_unstale":"2024-02-10T06:44:44.983450+0000","last_undegraded":"2024-02-10T06:44:44.983450+0000","last_fullsized":"2024-02-10T06:44:44.983450+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:39:32.171612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193441+0000","last_change":"2024-02-10T06:44:07.204235+0000","last_active":"2024-02-10T06:45:18.193441+0000","last_peered":"2024-02-10T06:45:18.193441+0000","last_clean":"2024-02-10T06:45:18.193441+0000","last_became_active":"2024-02-10T06:44:07.203514+0000","last_became_peered":"2024-02-10T06:44:07.203514+0000","last_unstale":"2024-02-10T06:45:18.193441+0000","last_undegraded":"2024-02-10T06:45:18.193441+0000","last_fullsized":"2024-02-10T06:45:18.193441+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:57:53.155613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869198+0000","last_change":"2024-02-10T06:44:09.210017+0000","last_active":"2024-02-10T06:44:44.869198+0000","last_peered":"2024-02-10T06:44:44.869198+0000","last_clean":"2024-02-10T06:44:44.869198+0000","last_became_active":"2024-02-10T06:44:09.209705+0000","last_became_peered":"2024-02-10T06:44:09.209705+0000","last_unstale":"2024-02-10T06:44:44.869198+0000","last_undegraded":"2024-02-10T06:44:44.869198+0000","last_fullsized":"2024-02-10T06:44:44.869198+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:29.690364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870077+0000","last_change":"2024-02-10T06:44:03.137411+0000","last_active":"2024-02-10T06:44:44.870077+0000","last_peered":"2024-02-10T06:44:44.870077+0000","last_clean":"2024-02-10T06:44:44.870077+0000","last_became_active":"2024-02-10T06:44:03.137124+0000","last_became_peered":"2024-02-10T06:44:03.137124+0000","last_unstale":"2024-02-10T06:44:44.870077+0000","last_undegraded":"2024-02-10T06:44:44.870077+0000","last_fullsized":"2024-02-10T06:44:44.870077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:28:36.069474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"67'28","reported_seq":94,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193441+0000","last_change":"2024-02-10T06:44:05.196676+0000","last_active":"2024-02-10T06:45:18.193441+0000","last_peered":"2024-02-10T06:45:18.193441+0000","last_clean":"2024-02-10T06:45:18.193441+0000","last_became_active":"2024-02-10T06:44:05.196381+0000","last_became_peered":"2024-02-10T06:44:05.196381+0000","last_unstale":"2024-02-10T06:45:18.193441+0000","last_undegraded":"2024-02-10T06:45:18.193441+0000","last_fullsized":"2024-02-10T06:45:18.193441+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:21:07.356318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867744+0000","last_change":"2024-02-10T06:44:07.203961+0000","last_active":"2024-02-10T06:44:44.867744+0000","last_peered":"2024-02-10T06:44:44.867744+0000","last_clean":"2024-02-10T06:44:44.867744+0000","last_became_active":"2024-02-10T06:44:07.203125+0000","last_became_peered":"2024-02-10T06:44:07.203125+0000","last_unstale":"2024-02-10T06:44:44.867744+0000","last_undegraded":"2024-02-10T06:44:44.867744+0000","last_fullsized":"2024-02-10T06:44:44.867744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:30:35.632812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495813+0000","last_change":"2024-02-10T06:44:09.215807+0000","last_active":"2024-02-10T06:44:45.495813+0000","last_peered":"2024-02-10T06:44:45.495813+0000","last_clean":"2024-02-10T06:44:45.495813+0000","last_became_active":"2024-02-10T06:44:09.215552+0000","last_became_peered":"2024-02-10T06:44:09.215552+0000","last_unstale":"2024-02-10T06:44:45.495813+0000","last_undegraded":"2024-02-10T06:44:45.495813+0000","last_fullsized":"2024-02-10T06:44:45.495813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:20:30.119417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"67'5","reported_seq":120,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:24.558659+0000","last_change":"2024-02-10T06:44:07.200823+0000","last_active":"2024-02-10T06:45:24.558659+0000","last_peered":"2024-02-10T06:45:24.558659+0000","last_clean":"2024-02-10T06:45:24.558659+0000","last_became_active":"2024-02-10T06:44:03.137370+0000","last_became_peered":"2024-02-10T06:44:03.137370+0000","last_unstale":"2024-02-10T06:45:24.558659+0000","last_undegraded":"2024-02-10T06:45:24.558659+0000","last_fullsized":"2024-02-10T06:45:24.558659+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:20:00.259539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049614910000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918786+0000","last_change":"2024-02-10T06:44:03.140681+0000","last_active":"2024-02-10T06:44:44.918786+0000","last_peered":"2024-02-10T06:44:44.918786+0000","last_clean":"2024-02-10T06:44:44.918786+0000","last_became_active":"2024-02-10T06:44:03.140442+0000","last_became_peered":"2024-02-10T06:44:03.140442+0000","last_unstale":"2024-02-10T06:44:44.918786+0000","last_undegraded":"2024-02-10T06:44:44.918786+0000","last_fullsized":"2024-02-10T06:44:44.918786+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:03:11.632597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984801+0000","last_change":"2024-02-10T06:44:05.197477+0000","last_active":"2024-02-10T06:44:44.984801+0000","last_peered":"2024-02-10T06:44:44.984801+0000","last_clean":"2024-02-10T06:44:44.984801+0000","last_became_active":"2024-02-10T06:44:05.197163+0000","last_became_peered":"2024-02-10T06:44:05.197163+0000","last_unstale":"2024-02-10T06:44:44.984801+0000","last_undegraded":"2024-02-10T06:44:44.984801+0000","last_fullsized":"2024-02-10T06:44:44.984801+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:35:28.665783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919088+0000","last_change":"2024-02-10T06:44:07.198598+0000","last_active":"2024-02-10T06:44:44.919088+0000","last_peered":"2024-02-10T06:44:44.919088+0000","last_clean":"2024-02-10T06:44:44.919088+0000","last_became_active":"2024-02-10T06:44:07.198304+0000","last_became_peered":"2024-02-10T06:44:07.198304+0000","last_unstale":"2024-02-10T06:44:44.919088+0000","last_undegraded":"2024-02-10T06:44:44.919088+0000","last_fullsized":"2024-02-10T06:44:44.919088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:19:30.304295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.871359+0000","last_change":"2024-02-10T06:44:09.215393+0000","last_active":"2024-02-10T06:44:44.871359+0000","last_peered":"2024-02-10T06:44:44.871359+0000","last_clean":"2024-02-10T06:44:44.871359+0000","last_became_active":"2024-02-10T06:44:09.215118+0000","last_became_peered":"2024-02-10T06:44:09.215118+0000","last_unstale":"2024-02-10T06:44:44.871359+0000","last_undegraded":"2024-02-10T06:44:44.871359+0000","last_fullsized":"2024-02-10T06:44:44.871359+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:30:47.762572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"64'2","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868155+0000","last_change":"2024-02-10T06:44:07.187187+0000","last_active":"2024-02-10T06:44:44.868155+0000","last_peered":"2024-02-10T06:44:44.868155+0000","last_clean":"2024-02-10T06:44:44.868155+0000","last_became_active":"2024-02-10T06:44:03.136155+0000","last_became_peered":"2024-02-10T06:44:03.136155+0000","last_unstale":"2024-02-10T06:44:44.868155+0000","last_undegraded":"2024-02-10T06:44:44.868155+0000","last_fullsized":"2024-02-10T06:44:44.868155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:31:40.233102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055916500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870241+0000","last_change":"2024-02-10T06:44:03.130646+0000","last_active":"2024-02-10T06:44:44.870241+0000","last_peered":"2024-02-10T06:44:44.870241+0000","last_clean":"2024-02-10T06:44:44.870241+0000","last_became_active":"2024-02-10T06:44:03.130367+0000","last_became_peered":"2024-02-10T06:44:03.130367+0000","last_unstale":"2024-02-10T06:44:44.870241+0000","last_undegraded":"2024-02-10T06:44:44.870241+0000","last_fullsized":"2024-02-10T06:44:44.870241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:02:53.864472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":276,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:46.968932+0000","last_change":"2024-02-10T06:43:53.469413+0000","last_active":"2024-02-10T06:44:46.968932+0000","last_peered":"2024-02-10T06:44:46.968932+0000","last_clean":"2024-02-10T06:44:46.968932+0000","last_became_active":"2024-02-10T06:43:53.468956+0000","last_became_peered":"2024-02-10T06:43:53.468956+0000","last_unstale":"2024-02-10T06:44:46.968932+0000","last_undegraded":"2024-02-10T06:44:46.968932+0000","last_fullsized":"2024-02-10T06:44:46.968932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:41:41.393633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:41:41.393633+0000","last_clean_scrub_stamp":"2024-02-10T06:41:41.393633+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:36:26.201690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'16","reported_seq":68,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193642+0000","last_change":"2024-02-10T06:44:05.332613+0000","last_active":"2024-02-10T06:45:18.193642+0000","last_peered":"2024-02-10T06:45:18.193642+0000","last_clean":"2024-02-10T06:45:18.193642+0000","last_became_active":"2024-02-10T06:44:05.332311+0000","last_became_peered":"2024-02-10T06:44:05.332311+0000","last_unstale":"2024-02-10T06:45:18.193642+0000","last_undegraded":"2024-02-10T06:45:18.193642+0000","last_fullsized":"2024-02-10T06:45:18.193642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:06:42.524661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495420+0000","last_change":"2024-02-10T06:44:07.209631+0000","last_active":"2024-02-10T06:44:45.495420+0000","last_peered":"2024-02-10T06:44:45.495420+0000","last_clean":"2024-02-10T06:44:45.495420+0000","last_became_active":"2024-02-10T06:44:07.209262+0000","last_became_peered":"2024-02-10T06:44:07.209262+0000","last_unstale":"2024-02-10T06:44:45.495420+0000","last_undegraded":"2024-02-10T06:44:45.495420+0000","last_fullsized":"2024-02-10T06:44:45.495420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:31:32.377481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983821+0000","last_change":"2024-02-10T06:44:09.208268+0000","last_active":"2024-02-10T06:44:44.983821+0000","last_peered":"2024-02-10T06:44:44.983821+0000","last_clean":"2024-02-10T06:44:44.983821+0000","last_became_active":"2024-02-10T06:44:09.207977+0000","last_became_peered":"2024-02-10T06:44:09.207977+0000","last_unstale":"2024-02-10T06:44:44.983821+0000","last_undegraded":"2024-02-10T06:44:44.983821+0000","last_fullsized":"2024-02-10T06:44:44.983821+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:33:34.844848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867199+0000","last_change":"2024-02-10T06:44:07.193931+0000","last_active":"2024-02-10T06:44:44.867199+0000","last_peered":"2024-02-10T06:44:44.867199+0000","last_clean":"2024-02-10T06:44:44.867199+0000","last_became_active":"2024-02-10T06:44:03.137349+0000","last_became_peered":"2024-02-10T06:44:03.137349+0000","last_unstale":"2024-02-10T06:44:44.867199+0000","last_undegraded":"2024-02-10T06:44:44.867199+0000","last_fullsized":"2024-02-10T06:44:44.867199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:00:24.379348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002247083,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494256+0000","last_change":"2024-02-10T06:44:03.139424+0000","last_active":"2024-02-10T06:44:45.494256+0000","last_peered":"2024-02-10T06:44:45.494256+0000","last_clean":"2024-02-10T06:44:45.494256+0000","last_became_active":"2024-02-10T06:44:03.139090+0000","last_became_peered":"2024-02-10T06:44:03.139090+0000","last_unstale":"2024-02-10T06:44:45.494256+0000","last_undegraded":"2024-02-10T06:44:45.494256+0000","last_fullsized":"2024-02-10T06:44:45.494256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:50:52.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868241+0000","last_change":"2024-02-10T06:44:05.195965+0000","last_active":"2024-02-10T06:44:44.868241+0000","last_peered":"2024-02-10T06:44:44.868241+0000","last_clean":"2024-02-10T06:44:44.868241+0000","last_became_active":"2024-02-10T06:44:05.195644+0000","last_became_peered":"2024-02-10T06:44:05.195644+0000","last_unstale":"2024-02-10T06:44:44.868241+0000","last_undegraded":"2024-02-10T06:44:44.868241+0000","last_fullsized":"2024-02-10T06:44:44.868241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:52:00.789709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.871274+0000","last_change":"2024-02-10T06:44:07.202091+0000","last_active":"2024-02-10T06:44:44.871274+0000","last_peered":"2024-02-10T06:44:44.871274+0000","last_clean":"2024-02-10T06:44:44.871274+0000","last_became_active":"2024-02-10T06:44:07.201779+0000","last_became_peered":"2024-02-10T06:44:07.201779+0000","last_unstale":"2024-02-10T06:44:44.871274+0000","last_undegraded":"2024-02-10T06:44:44.871274+0000","last_fullsized":"2024-02-10T06:44:44.871274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:28:16.486258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983110+0000","last_change":"2024-02-10T06:44:09.209280+0000","last_active":"2024-02-10T06:44:44.983110+0000","last_peered":"2024-02-10T06:44:44.983110+0000","last_clean":"2024-02-10T06:44:44.983110+0000","last_became_active":"2024-02-10T06:44:09.208958+0000","last_became_peered":"2024-02-10T06:44:09.208958+0000","last_unstale":"2024-02-10T06:44:44.983110+0000","last_undegraded":"2024-02-10T06:44:44.983110+0000","last_fullsized":"2024-02-10T06:44:44.983110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:07:52.245853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495899+0000","last_change":"2024-02-10T06:44:03.133479+0000","last_active":"2024-02-10T06:44:45.495899+0000","last_peered":"2024-02-10T06:44:45.495899+0000","last_clean":"2024-02-10T06:44:45.495899+0000","last_became_active":"2024-02-10T06:44:03.133226+0000","last_became_peered":"2024-02-10T06:44:03.133226+0000","last_unstale":"2024-02-10T06:44:45.495899+0000","last_undegraded":"2024-02-10T06:44:45.495899+0000","last_fullsized":"2024-02-10T06:44:45.495899+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:01:35.101773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867287+0000","last_change":"2024-02-10T06:44:05.332232+0000","last_active":"2024-02-10T06:44:44.867287+0000","last_peered":"2024-02-10T06:44:44.867287+0000","last_clean":"2024-02-10T06:44:44.867287+0000","last_became_active":"2024-02-10T06:44:05.331961+0000","last_became_peered":"2024-02-10T06:44:05.331961+0000","last_unstale":"2024-02-10T06:44:44.867287+0000","last_undegraded":"2024-02-10T06:44:44.867287+0000","last_fullsized":"2024-02-10T06:44:44.867287+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:22:19.929930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496041+0000","last_change":"2024-02-10T06:44:07.199135+0000","last_active":"2024-02-10T06:44:45.496041+0000","last_peered":"2024-02-10T06:44:45.496041+0000","last_clean":"2024-02-10T06:44:45.496041+0000","last_became_active":"2024-02-10T06:44:07.198757+0000","last_became_peered":"2024-02-10T06:44:07.198757+0000","last_unstale":"2024-02-10T06:44:45.496041+0000","last_undegraded":"2024-02-10T06:44:45.496041+0000","last_fullsized":"2024-02-10T06:44:45.496041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:21:24.785285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494551+0000","last_change":"2024-02-10T06:44:09.207729+0000","last_active":"2024-02-10T06:44:45.494551+0000","last_peered":"2024-02-10T06:44:45.494551+0000","last_clean":"2024-02-10T06:44:45.494551+0000","last_became_active":"2024-02-10T06:44:09.207432+0000","last_became_peered":"2024-02-10T06:44:09.207432+0000","last_unstale":"2024-02-10T06:44:45.494551+0000","last_undegraded":"2024-02-10T06:44:45.494551+0000","last_fullsized":"2024-02-10T06:44:45.494551+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:58:32.708616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"60'1","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984432+0000","last_change":"2024-02-10T06:44:03.137221+0000","last_active":"2024-02-10T06:44:44.984432+0000","last_peered":"2024-02-10T06:44:44.984432+0000","last_clean":"2024-02-10T06:44:44.984432+0000","last_became_active":"2024-02-10T06:44:03.136716+0000","last_became_peered":"2024-02-10T06:44:03.136716+0000","last_unstale":"2024-02-10T06:44:44.984432+0000","last_undegraded":"2024-02-10T06:44:44.984432+0000","last_fullsized":"2024-02-10T06:44:44.984432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:38:20.089065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867839+0000","last_change":"2024-02-10T06:44:05.332423+0000","last_active":"2024-02-10T06:44:44.867839+0000","last_peered":"2024-02-10T06:44:44.867839+0000","last_clean":"2024-02-10T06:44:44.867839+0000","last_became_active":"2024-02-10T06:44:05.332171+0000","last_became_peered":"2024-02-10T06:44:05.332171+0000","last_unstale":"2024-02-10T06:44:44.867839+0000","last_undegraded":"2024-02-10T06:44:44.867839+0000","last_fullsized":"2024-02-10T06:44:44.867839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:52:42.865648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494106+0000","last_change":"2024-02-10T06:44:07.201321+0000","last_active":"2024-02-10T06:44:45.494106+0000","last_peered":"2024-02-10T06:44:45.494106+0000","last_clean":"2024-02-10T06:44:45.494106+0000","last_became_active":"2024-02-10T06:44:07.200961+0000","last_became_peered":"2024-02-10T06:44:07.200961+0000","last_unstale":"2024-02-10T06:44:45.494106+0000","last_undegraded":"2024-02-10T06:44:45.494106+0000","last_fullsized":"2024-02-10T06:44:45.494106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:47:08.986191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"67'1","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495463+0000","last_change":"2024-02-10T06:44:09.213677+0000","last_active":"2024-02-10T06:44:45.495463+0000","last_peered":"2024-02-10T06:44:45.495463+0000","last_clean":"2024-02-10T06:44:45.495463+0000","last_became_active":"2024-02-10T06:44:09.213405+0000","last_became_peered":"2024-02-10T06:44:09.213405+0000","last_unstale":"2024-02-10T06:44:45.495463+0000","last_undegraded":"2024-02-10T06:44:45.495463+0000","last_fullsized":"2024-02-10T06:44:45.495463+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:16:35.414457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496478+0000","last_change":"2024-02-10T06:44:03.133961+0000","last_active":"2024-02-10T06:44:45.496478+0000","last_peered":"2024-02-10T06:44:45.496478+0000","last_clean":"2024-02-10T06:44:45.496478+0000","last_became_active":"2024-02-10T06:44:03.133579+0000","last_became_peered":"2024-02-10T06:44:03.133579+0000","last_unstale":"2024-02-10T06:44:45.496478+0000","last_undegraded":"2024-02-10T06:44:45.496478+0000","last_fullsized":"2024-02-10T06:44:45.496478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:16:55.679227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493839+0000","last_change":"2024-02-10T06:44:05.192023+0000","last_active":"2024-02-10T06:44:45.493839+0000","last_peered":"2024-02-10T06:44:45.493839+0000","last_clean":"2024-02-10T06:44:45.493839+0000","last_became_active":"2024-02-10T06:44:05.191763+0000","last_became_peered":"2024-02-10T06:44:05.191763+0000","last_unstale":"2024-02-10T06:44:45.493839+0000","last_undegraded":"2024-02-10T06:44:45.493839+0000","last_fullsized":"2024-02-10T06:44:45.493839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:51:44.991293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193444+0000","last_change":"2024-02-10T06:44:07.202437+0000","last_active":"2024-02-10T06:45:18.193444+0000","last_peered":"2024-02-10T06:45:18.193444+0000","last_clean":"2024-02-10T06:45:18.193444+0000","last_became_active":"2024-02-10T06:44:07.202107+0000","last_became_peered":"2024-02-10T06:44:07.202107+0000","last_unstale":"2024-02-10T06:45:18.193444+0000","last_undegraded":"2024-02-10T06:45:18.193444+0000","last_fullsized":"2024-02-10T06:45:18.193444+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:35:59.282975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868216+0000","last_change":"2024-02-10T06:44:09.208220+0000","last_active":"2024-02-10T06:44:44.868216+0000","last_peered":"2024-02-10T06:44:44.868216+0000","last_clean":"2024-02-10T06:44:44.868216+0000","last_became_active":"2024-02-10T06:44:09.207939+0000","last_became_peered":"2024-02-10T06:44:09.207939+0000","last_unstale":"2024-02-10T06:44:44.868216+0000","last_undegraded":"2024-02-10T06:44:44.868216+0000","last_fullsized":"2024-02-10T06:44:44.868216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:07:16.019437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.871191+0000","last_change":"2024-02-10T06:44:07.200370+0000","last_active":"2024-02-10T06:44:44.871191+0000","last_peered":"2024-02-10T06:44:44.871191+0000","last_clean":"2024-02-10T06:44:44.871191+0000","last_became_active":"2024-02-10T06:44:07.200049+0000","last_became_peered":"2024-02-10T06:44:07.200049+0000","last_unstale":"2024-02-10T06:44:44.871191+0000","last_undegraded":"2024-02-10T06:44:44.871191+0000","last_fullsized":"2024-02-10T06:44:44.871191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:09:42.898197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494447+0000","last_change":"2024-02-10T06:44:03.136260+0000","last_active":"2024-02-10T06:44:45.494447+0000","last_peered":"2024-02-10T06:44:45.494447+0000","last_clean":"2024-02-10T06:44:45.494447+0000","last_became_active":"2024-02-10T06:44:03.135873+0000","last_became_peered":"2024-02-10T06:44:03.135873+0000","last_unstale":"2024-02-10T06:44:45.494447+0000","last_undegraded":"2024-02-10T06:44:45.494447+0000","last_fullsized":"2024-02-10T06:44:45.494447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:16.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493503+0000","last_change":"2024-02-10T06:44:05.198080+0000","last_active":"2024-02-10T06:44:45.493503+0000","last_peered":"2024-02-10T06:44:45.493503+0000","last_clean":"2024-02-10T06:44:45.493503+0000","last_became_active":"2024-02-10T06:44:05.197796+0000","last_became_peered":"2024-02-10T06:44:05.197796+0000","last_unstale":"2024-02-10T06:44:45.493503+0000","last_undegraded":"2024-02-10T06:44:45.493503+0000","last_fullsized":"2024-02-10T06:44:45.493503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:32:47.398389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870529+0000","last_change":"2024-02-10T06:44:09.213948+0000","last_active":"2024-02-10T06:44:44.870529+0000","last_peered":"2024-02-10T06:44:44.870529+0000","last_clean":"2024-02-10T06:44:44.870529+0000","last_became_active":"2024-02-10T06:44:09.213547+0000","last_became_peered":"2024-02-10T06:44:09.213547+0000","last_unstale":"2024-02-10T06:44:44.870529+0000","last_undegraded":"2024-02-10T06:44:44.870529+0000","last_fullsized":"2024-02-10T06:44:44.870529+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:15:53.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193626+0000","last_change":"2024-02-10T06:44:07.198041+0000","last_active":"2024-02-10T06:45:18.193626+0000","last_peered":"2024-02-10T06:45:18.193626+0000","last_clean":"2024-02-10T06:45:18.193626+0000","last_became_active":"2024-02-10T06:44:07.197737+0000","last_became_peered":"2024-02-10T06:44:07.197737+0000","last_unstale":"2024-02-10T06:45:18.193626+0000","last_undegraded":"2024-02-10T06:45:18.193626+0000","last_fullsized":"2024-02-10T06:45:18.193626+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:49:22.956013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983303+0000","last_change":"2024-02-10T06:44:03.139934+0000","last_active":"2024-02-10T06:44:44.983303+0000","last_peered":"2024-02-10T06:44:44.983303+0000","last_clean":"2024-02-10T06:44:44.983303+0000","last_became_active":"2024-02-10T06:44:03.134239+0000","last_became_peered":"2024-02-10T06:44:03.134239+0000","last_unstale":"2024-02-10T06:44:44.983303+0000","last_undegraded":"2024-02-10T06:44:44.983303+0000","last_fullsized":"2024-02-10T06:44:44.983303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:21:03.100156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919208+0000","last_change":"2024-02-10T06:44:23.612907+0000","last_active":"2024-02-10T06:44:44.919208+0000","last_peered":"2024-02-10T06:44:44.919208+0000","last_clean":"2024-02-10T06:44:44.919208+0000","last_became_active":"2024-02-10T06:44:23.601024+0000","last_became_peered":"2024-02-10T06:44:23.601024+0000","last_unstale":"2024-02-10T06:44:44.919208+0000","last_undegraded":"2024-02-10T06:44:44.919208+0000","last_fullsized":"2024-02-10T06:44:44.919208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:03:23.937127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495378+0000","last_change":"2024-02-10T06:44:09.211740+0000","last_active":"2024-02-10T06:44:45.495378+0000","last_peered":"2024-02-10T06:44:45.495378+0000","last_clean":"2024-02-10T06:44:45.495378+0000","last_became_active":"2024-02-10T06:44:09.211444+0000","last_became_peered":"2024-02-10T06:44:09.211444+0000","last_unstale":"2024-02-10T06:44:45.495378+0000","last_undegraded":"2024-02-10T06:44:45.495378+0000","last_fullsized":"2024-02-10T06:44:45.495378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:05:18.228024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193445+0000","last_change":"2024-02-10T06:44:07.197665+0000","last_active":"2024-02-10T06:45:18.193445+0000","last_peered":"2024-02-10T06:45:18.193445+0000","last_clean":"2024-02-10T06:45:18.193445+0000","last_became_active":"2024-02-10T06:44:07.197348+0000","last_became_peered":"2024-02-10T06:44:07.197348+0000","last_unstale":"2024-02-10T06:45:18.193445+0000","last_undegraded":"2024-02-10T06:45:18.193445+0000","last_fullsized":"2024-02-10T06:45:18.193445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:48:34.244194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983351+0000","last_change":"2024-02-10T06:44:03.134744+0000","last_active":"2024-02-10T06:44:44.983351+0000","last_peered":"2024-02-10T06:44:44.983351+0000","last_clean":"2024-02-10T06:44:44.983351+0000","last_became_active":"2024-02-10T06:44:03.134431+0000","last_became_peered":"2024-02-10T06:44:03.134431+0000","last_unstale":"2024-02-10T06:44:44.983351+0000","last_undegraded":"2024-02-10T06:44:44.983351+0000","last_fullsized":"2024-02-10T06:44:44.983351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:07:24.103517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493659+0000","last_change":"2024-02-10T06:44:05.192829+0000","last_active":"2024-02-10T06:44:45.493659+0000","last_peered":"2024-02-10T06:44:45.493659+0000","last_clean":"2024-02-10T06:44:45.493659+0000","last_became_active":"2024-02-10T06:44:05.192543+0000","last_became_peered":"2024-02-10T06:44:05.192543+0000","last_unstale":"2024-02-10T06:44:45.493659+0000","last_undegraded":"2024-02-10T06:44:45.493659+0000","last_fullsized":"2024-02-10T06:44:45.493659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:21:26.897748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919264+0000","last_change":"2024-02-10T06:44:09.209054+0000","last_active":"2024-02-10T06:44:44.919264+0000","last_peered":"2024-02-10T06:44:44.919264+0000","last_clean":"2024-02-10T06:44:44.919264+0000","last_became_active":"2024-02-10T06:44:09.208534+0000","last_became_peered":"2024-02-10T06:44:09.208534+0000","last_unstale":"2024-02-10T06:44:44.919264+0000","last_undegraded":"2024-02-10T06:44:44.919264+0000","last_fullsized":"2024-02-10T06:44:44.919264+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:17:47.646864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983909+0000","last_change":"2024-02-10T06:44:07.209886+0000","last_active":"2024-02-10T06:44:44.983909+0000","last_peered":"2024-02-10T06:44:44.983909+0000","last_clean":"2024-02-10T06:44:44.983909+0000","last_became_active":"2024-02-10T06:44:07.209393+0000","last_became_peered":"2024-02-10T06:44:07.209393+0000","last_unstale":"2024-02-10T06:44:44.983909+0000","last_undegraded":"2024-02-10T06:44:44.983909+0000","last_fullsized":"2024-02-10T06:44:44.983909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:33:08.994086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495131+0000","last_change":"2024-02-10T06:44:03.134895+0000","last_active":"2024-02-10T06:44:45.495131+0000","last_peered":"2024-02-10T06:44:45.495131+0000","last_clean":"2024-02-10T06:44:45.495131+0000","last_became_active":"2024-02-10T06:44:03.134398+0000","last_became_peered":"2024-02-10T06:44:03.134398+0000","last_unstale":"2024-02-10T06:44:45.495131+0000","last_undegraded":"2024-02-10T06:44:45.495131+0000","last_fullsized":"2024-02-10T06:44:45.495131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:16:29.819393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493476+0000","last_change":"2024-02-10T06:44:05.198764+0000","last_active":"2024-02-10T06:44:45.493476+0000","last_peered":"2024-02-10T06:44:45.493476+0000","last_clean":"2024-02-10T06:44:45.493476+0000","last_became_active":"2024-02-10T06:44:05.198450+0000","last_became_peered":"2024-02-10T06:44:05.198450+0000","last_unstale":"2024-02-10T06:44:45.493476+0000","last_undegraded":"2024-02-10T06:44:45.493476+0000","last_fullsized":"2024-02-10T06:44:45.493476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:12:12.836875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494163+0000","last_change":"2024-02-10T06:44:09.213430+0000","last_active":"2024-02-10T06:44:45.494163+0000","last_peered":"2024-02-10T06:44:45.494163+0000","last_clean":"2024-02-10T06:44:45.494163+0000","last_became_active":"2024-02-10T06:44:09.212659+0000","last_became_peered":"2024-02-10T06:44:09.212659+0000","last_unstale":"2024-02-10T06:44:45.494163+0000","last_undegraded":"2024-02-10T06:44:45.494163+0000","last_fullsized":"2024-02-10T06:44:45.494163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:48:39.295872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918995+0000","last_change":"2024-02-10T06:44:07.195008+0000","last_active":"2024-02-10T06:44:44.918995+0000","last_peered":"2024-02-10T06:44:44.918995+0000","last_clean":"2024-02-10T06:44:44.918995+0000","last_became_active":"2024-02-10T06:44:07.194659+0000","last_became_peered":"2024-02-10T06:44:07.194659+0000","last_unstale":"2024-02-10T06:44:44.918995+0000","last_undegraded":"2024-02-10T06:44:44.918995+0000","last_fullsized":"2024-02-10T06:44:44.918995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:45:45.411029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919270+0000","last_change":"2024-02-10T06:44:03.139848+0000","last_active":"2024-02-10T06:44:44.919270+0000","last_peered":"2024-02-10T06:44:44.919270+0000","last_clean":"2024-02-10T06:44:44.919270+0000","last_became_active":"2024-02-10T06:44:03.139568+0000","last_became_peered":"2024-02-10T06:44:03.139568+0000","last_unstale":"2024-02-10T06:44:44.919270+0000","last_undegraded":"2024-02-10T06:44:44.919270+0000","last_fullsized":"2024-02-10T06:44:44.919270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:29:09.134748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495768+0000","last_change":"2024-02-10T06:44:05.331722+0000","last_active":"2024-02-10T06:44:45.495768+0000","last_peered":"2024-02-10T06:44:45.495768+0000","last_clean":"2024-02-10T06:44:45.495768+0000","last_became_active":"2024-02-10T06:44:05.331226+0000","last_became_peered":"2024-02-10T06:44:05.331226+0000","last_unstale":"2024-02-10T06:44:45.495768+0000","last_undegraded":"2024-02-10T06:44:45.495768+0000","last_fullsized":"2024-02-10T06:44:45.495768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:29:42.502340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.866894+0000","last_change":"2024-02-10T06:44:09.206911+0000","last_active":"2024-02-10T06:44:44.866894+0000","last_peered":"2024-02-10T06:44:44.866894+0000","last_clean":"2024-02-10T06:44:44.866894+0000","last_became_active":"2024-02-10T06:44:09.206602+0000","last_became_peered":"2024-02-10T06:44:09.206602+0000","last_unstale":"2024-02-10T06:44:44.866894+0000","last_undegraded":"2024-02-10T06:44:44.866894+0000","last_fullsized":"2024-02-10T06:44:44.866894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:54:55.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918948+0000","last_change":"2024-02-10T06:44:07.195594+0000","last_active":"2024-02-10T06:44:44.918948+0000","last_peered":"2024-02-10T06:44:44.918948+0000","last_clean":"2024-02-10T06:44:44.918948+0000","last_became_active":"2024-02-10T06:44:07.195286+0000","last_became_peered":"2024-02-10T06:44:07.195286+0000","last_unstale":"2024-02-10T06:44:44.918948+0000","last_undegraded":"2024-02-10T06:44:44.918948+0000","last_fullsized":"2024-02-10T06:44:44.918948+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:42:52.871553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"60'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984514+0000","last_change":"2024-02-10T06:44:03.136088+0000","last_active":"2024-02-10T06:44:44.984514+0000","last_peered":"2024-02-10T06:44:44.984514+0000","last_clean":"2024-02-10T06:44:44.984514+0000","last_became_active":"2024-02-10T06:44:03.134812+0000","last_became_peered":"2024-02-10T06:44:03.134812+0000","last_unstale":"2024-02-10T06:44:44.984514+0000","last_undegraded":"2024-02-10T06:44:44.984514+0000","last_fullsized":"2024-02-10T06:44:44.984514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:37.446247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867432+0000","last_change":"2024-02-10T06:44:05.196308+0000","last_active":"2024-02-10T06:44:44.867432+0000","last_peered":"2024-02-10T06:44:44.867432+0000","last_clean":"2024-02-10T06:44:44.867432+0000","last_became_active":"2024-02-10T06:44:05.196019+0000","last_became_peered":"2024-02-10T06:44:05.196019+0000","last_unstale":"2024-02-10T06:44:44.867432+0000","last_undegraded":"2024-02-10T06:44:44.867432+0000","last_fullsized":"2024-02-10T06:44:44.867432+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:58.667363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869890+0000","last_change":"2024-02-10T06:44:09.209515+0000","last_active":"2024-02-10T06:44:44.869890+0000","last_peered":"2024-02-10T06:44:44.869890+0000","last_clean":"2024-02-10T06:44:44.869890+0000","last_became_active":"2024-02-10T06:44:09.209202+0000","last_became_peered":"2024-02-10T06:44:09.209202+0000","last_unstale":"2024-02-10T06:44:44.869890+0000","last_undegraded":"2024-02-10T06:44:44.869890+0000","last_fullsized":"2024-02-10T06:44:44.869890+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:58:28.030431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919277+0000","last_change":"2024-02-10T06:44:03.137324+0000","last_active":"2024-02-10T06:44:44.919277+0000","last_peered":"2024-02-10T06:44:44.919277+0000","last_clean":"2024-02-10T06:44:44.919277+0000","last_became_active":"2024-02-10T06:44:03.136986+0000","last_became_peered":"2024-02-10T06:44:03.136986+0000","last_unstale":"2024-02-10T06:44:44.919277+0000","last_undegraded":"2024-02-10T06:44:44.919277+0000","last_fullsized":"2024-02-10T06:44:44.919277+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:53:06.162086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869856+0000","last_change":"2024-02-10T06:44:05.333020+0000","last_active":"2024-02-10T06:44:44.869856+0000","last_peered":"2024-02-10T06:44:44.869856+0000","last_clean":"2024-02-10T06:44:44.869856+0000","last_became_active":"2024-02-10T06:44:05.332656+0000","last_became_peered":"2024-02-10T06:44:05.332656+0000","last_unstale":"2024-02-10T06:44:44.869856+0000","last_undegraded":"2024-02-10T06:44:44.869856+0000","last_fullsized":"2024-02-10T06:44:44.869856+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:50:15.802954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:18.193623+0000","last_change":"2024-02-10T06:44:07.203447+0000","last_active":"2024-02-10T06:45:18.193623+0000","last_peered":"2024-02-10T06:45:18.193623+0000","last_clean":"2024-02-10T06:45:18.193623+0000","last_became_active":"2024-02-10T06:44:07.203138+0000","last_became_peered":"2024-02-10T06:44:07.203138+0000","last_unstale":"2024-02-10T06:45:18.193623+0000","last_undegraded":"2024-02-10T06:45:18.193623+0000","last_fullsized":"2024-02-10T06:45:18.193623+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:03:16.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234006,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32444,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712708,"statfs":{"total":95995035648,"available":95961812992,"internally_reserved":0,"allocated":1298432,"data_stored":867391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.621},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63500000000000001}]},{"osd":3,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.41299999999999998}]},{"osd":5,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.56699999999999995}]},{"osd":6,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.496}]}]},{"osd":6,"up_from":46,"seq":197568495643,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32440,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712712,"statfs":{"total":95995035648,"available":95961817088,"internally_reserved":0,"allocated":1294336,"data_stored":867210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.35799999999999998}]},{"osd":1,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61399999999999999}]},{"osd":4,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.63200000000000001}]},{"osd":5,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.67200000000000004}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":733184,"data_stored":278136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.32900000000000001},"min":{"1min":0.23400000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57999999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53700000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.32500000000000001}]},{"osd":2,"last update":"Sat Feb 10 06:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.626,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38700000000000001}]},{"osd":3,"last update":"Sat Feb 10 06:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.318,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.1539999999999999,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47899999999999998,"15min":0.45500000000000002},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.313}]},{"osd":4,"last update":"Sat Feb 10 06:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Feb 10 06:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.434},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45300000000000001}]},{"osd":6,"last update":"Sat Feb 10 06:44:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40300000000000002}]},{"osd":7,"last update":"Sat Feb 10 06:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.48699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32448,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712704,"statfs":{"total":95995035648,"available":95961808896,"internally_reserved":0,"allocated":1302528,"data_stored":867995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 06:45:20 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36699999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.55200000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.55300000000000005}]},{"osd":2,"last update":"Sat Feb 10 06:44:48 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.41799999999999998,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61699999999999999,"5min":0.621,"15min":0.621},"last":0.496},{"interface":"front","average":{"1min":0.433,"5min":0.39600000000000002,"15min":0.38800000000000001},"min":{"1min":0.26600000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Sat Feb 10 06:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42499999999999999,"15min":0.42599999999999999},"min":{"1min":0.29099999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59899999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Feb 10 06:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75800000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Sat Feb 10 06:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60999999999999999}]},{"osd":6,"last update":"Sat Feb 10 06:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.75,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Sat Feb 10 06:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.621}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":696320,"data_stored":277084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35499999999999998}]},{"osd":1,"last update":"Sat Feb 10 06:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57599999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.377}]},{"osd":3,"last update":"Sat Feb 10 06:45:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42499999999999999,"15min":0.42299999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53500000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45300000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.41799999999999998}]},{"osd":4,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.752,"15min":0.752},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Sat Feb 10 06:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.873,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54100000000000004}]},{"osd":6,"last update":"Sat Feb 10 06:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.56499999999999995}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58999999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182443,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":278530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:45:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.41099999999999998,"15min":0.41699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.57899999999999996}]},{"osd":1,"last update":"Sat Feb 10 06:45:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.41599999999999998,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Feb 10 06:45:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.433,"15min":0.441},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68899999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.63}]},{"osd":4,"last update":"Sat Feb 10 06:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.70499999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.70699999999999996}]},{"osd":5,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":0.751,"15min":0.751},"last":0.75700000000000001}]},{"osd":6,"last update":"Sat Feb 10 06:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Feb 10 06:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.64000000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":278179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60099999999999998}]},{"osd":1,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73099999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69499999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61799999999999999}]},{"osd":3,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Feb 10 06:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.746,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41899999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76500000000000001}]},{"osd":6,"last update":"Sat Feb 10 06:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81899999999999995}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.80200000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757281,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713296,"statfs":{"total":95995035648,"available":95962415104,"internally_reserved":0,"allocated":692224,"data_stored":276783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.61899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.69299999999999995}]},{"osd":1,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.501,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.621,"5min":0.627,"15min":0.627},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.65200000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.53400000000000003}]},{"osd":2,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.751},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53500000000000003,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.51100000000000001}]},{"osd":3,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.54900000000000004}]},{"osd":6,"last update":"Sat Feb 10 06:44:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T06:45:31.555 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T06:45:31.795 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:32.205 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:32.205 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.205 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[29878]: Deploying daemon grafana.a on smithi184 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:31 smithi174 ceph-mon[39523]: Deploying daemon grafana.a on smithi184 2024-02-10T06:45:32.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:31 smithi184 ceph-mon[33513]: Deploying daemon grafana.a on smithi184 2024-02-10T06:45:32.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:32 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:32.793Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000422141s 2024-02-10T06:45:33.565 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 -- 172.21.15.174:0/3850798667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 msgr2=0x7f97041012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:33.565 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 --2- 172.21.15.174:0/3850798667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 0x7f97041012c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009990 tx=0x7f96f4009ca0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.565 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 -- 172.21.15.174:0/3850798667 shutdown_connections 2024-02-10T06:45:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 --2- 172.21.15.174:0/3850798667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704105f30 0x7f970410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 --2- 172.21.15.174:0/3850798667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 0x7f97041012c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 --2- 172.21.15.174:0/3850798667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704101890 0x7f9704105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 -- 172.21.15.174:0/3850798667 >> 172.21.15.174:0/3850798667 conn(0x7f9704076c70 msgr2=0x7f9704077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 -- 172.21.15.174:0/3850798667 shutdown_connections 2024-02-10T06:45:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.564+0000 7f9708f69700 1 -- 172.21.15.174:0/3850798667 wait complete. 2024-02-10T06:45:33.567 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.565+0000 7f9708f69700 1 Processor -- start 2024-02-10T06:45:33.567 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.565+0000 7f9708f69700 1 -- start start 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9708f69700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 0x7f9704195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9708f69700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704101890 0x7f9704195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9708f69700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 0x7f970419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9708f69700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970419a630 con 0x7f9704100ee0 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9708f69700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970419a7a0 con 0x7f9704105f30 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9708f69700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970419a960 con 0x7f9704101890 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 0x7f970419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 0x7f970419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.174:52830/0 (socket says 172.21.15.174:52830) 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 -- 172.21.15.174:0/3144599353 learned_addr learned my addr 172.21.15.174:0/3144599353 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f970259c700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 0x7f9704195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9701d9b700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704101890 0x7f9704195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 -- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704101890 msgr2=0x7f9704195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704101890 0x7f9704195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 -- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 msgr2=0x7f9704195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:33.568 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 0x7f9704195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 -- 172.21.15.174:0/3144599353 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc009c40 con 0x7f9704105f30 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9701d9b700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704101890 0x7f9704195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f9702d9d700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 0x7f970419a020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00c370 tx=0x7f96fc00c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.566+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc017070 con 0x7f9704105f30 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.567+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f40095f0 con 0x7f9704105f30 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.567+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f96fc00f040 con 0x7f9704105f30 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.567+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00e050 con 0x7f9704105f30 2024-02-10T06:45:33.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.567+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97041a22d0 con 0x7f9704105f30 2024-02-10T06:45:33.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.567+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97041091f0 con 0x7f9704105f30 2024-02-10T06:45:33.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.568+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f96fc004470 con 0x7f9704105f30 2024-02-10T06:45:33.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.568+0000 7f96f37fe700 1 --2- 172.21.15.174:0/3144599353 >> v2:172.21.15.174:6800/1252702393 conn(0x7f96ec06fc90 0x7f96ec072150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:33.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.568+0000 7f970259c700 1 --2- 172.21.15.174:0/3144599353 >> v2:172.21.15.174:6800/1252702393 conn(0x7f96ec06fc90 0x7f96ec072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:33.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.568+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f96fc08cd70 con 0x7f9704105f30 2024-02-10T06:45:33.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.569+0000 7f970259c700 1 --2- 172.21.15.174:0/3144599353 >> v2:172.21.15.174:6800/1252702393 conn(0x7f96ec06fc90 0x7f96ec072150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96f40098e0 tx=0x7f96f4005960 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:33.574 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.573+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96fc05a180 con 0x7f9704105f30 2024-02-10T06:45:33.763 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.762+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 --> v2:172.21.15.174:6800/1252702393 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9704196ab0 con 0x7f96ec06fc90 2024-02-10T06:45:33.774 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.773+0000 7f96f37fe700 1 -- 172.21.15.174:0/3144599353 <== mgr.14568 v2:172.21.15.174:6800/1252702393 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384235 (secure 0 0 0) 0x7f9704196ab0 con 0x7f96ec06fc90 2024-02-10T06:45:33.779 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 >> v2:172.21.15.174:6800/1252702393 conn(0x7f96ec06fc90 msgr2=0x7f96ec072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 --2- 172.21.15.174:0/3144599353 >> v2:172.21.15.174:6800/1252702393 conn(0x7f96ec06fc90 0x7f96ec072150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96f40098e0 tx=0x7f96f4005960 comp rx=0 tx=0).stop 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 msgr2=0x7f970419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 0x7f970419a020 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00c370 tx=0x7f96fc00c680 comp rx=0 tx=0).stop 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 shutdown_connections 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 --2- 172.21.15.174:0/3144599353 >> v2:172.21.15.174:6800/1252702393 conn(0x7f96ec06fc90 0x7f96ec072150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9704105f30 0x7f970419a020 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9704100ee0 0x7f9704195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 --2- 172.21.15.174:0/3144599353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9704101890 0x7f9704195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 >> 172.21.15.174:0/3144599353 conn(0x7f9704076c70 msgr2=0x7f9704100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:33.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 shutdown_connections 2024-02-10T06:45:33.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:33.776+0000 7f9708f69700 1 -- 172.21.15.174:0/3144599353 wait complete. 2024-02-10T06:45:33.783 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2024-02-10T06:45:34.132 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:33 smithi174 ceph-mon[29878]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:34.133 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:33 smithi174 ceph-mon[29878]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:45:34.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:33 smithi174 ceph-mon[39523]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:34.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:33 smithi174 ceph-mon[39523]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:45:34.174 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:33 smithi184 ceph-mon[33513]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:34.174 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:33 smithi184 ceph-mon[33513]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T06:45:34.503 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-02-10T06:45:32.888947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1016,"num_read_kb":862,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244240,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716976,"statfs":{"total":767960285184,"available":767710183424,"internally_reserved":0,"allocated":7467008,"data_stored":3991308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868368+0000","last_change":"2024-02-10T06:44:09.206711+0000","last_active":"2024-02-10T06:44:44.868368+0000","last_peered":"2024-02-10T06:44:44.868368+0000","last_clean":"2024-02-10T06:44:44.868368+0000","last_became_active":"2024-02-10T06:44:09.206417+0000","last_became_peered":"2024-02-10T06:44:09.206417+0000","last_unstale":"2024-02-10T06:44:44.868368+0000","last_undegraded":"2024-02-10T06:44:44.868368+0000","last_fullsized":"2024-02-10T06:44:44.868368+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:54:03.472311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493645+0000","last_change":"2024-02-10T06:44:07.206388+0000","last_active":"2024-02-10T06:44:45.493645+0000","last_peered":"2024-02-10T06:44:45.493645+0000","last_clean":"2024-02-10T06:44:45.493645+0000","last_became_active":"2024-02-10T06:44:07.206014+0000","last_became_peered":"2024-02-10T06:44:07.206014+0000","last_unstale":"2024-02-10T06:44:45.493645+0000","last_undegraded":"2024-02-10T06:44:45.493645+0000","last_fullsized":"2024-02-10T06:44:45.493645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:50:03.440192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868007+0000","last_change":"2024-02-10T06:44:03.133327+0000","last_active":"2024-02-10T06:44:44.868007+0000","last_peered":"2024-02-10T06:44:44.868007+0000","last_clean":"2024-02-10T06:44:44.868007+0000","last_became_active":"2024-02-10T06:44:03.132905+0000","last_became_peered":"2024-02-10T06:44:03.132905+0000","last_unstale":"2024-02-10T06:44:44.868007+0000","last_undegraded":"2024-02-10T06:44:44.868007+0000","last_fullsized":"2024-02-10T06:44:44.868007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:10:20.618882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868162+0000","last_change":"2024-02-10T06:44:05.190362+0000","last_active":"2024-02-10T06:44:44.868162+0000","last_peered":"2024-02-10T06:44:44.868162+0000","last_clean":"2024-02-10T06:44:44.868162+0000","last_became_active":"2024-02-10T06:44:05.190049+0000","last_became_peered":"2024-02-10T06:44:05.190049+0000","last_unstale":"2024-02-10T06:44:44.868162+0000","last_undegraded":"2024-02-10T06:44:44.868162+0000","last_fullsized":"2024-02-10T06:44:44.868162+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:37.660919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494258+0000","last_change":"2024-02-10T06:44:09.215857+0000","last_active":"2024-02-10T06:44:45.494258+0000","last_peered":"2024-02-10T06:44:45.494258+0000","last_clean":"2024-02-10T06:44:45.494258+0000","last_became_active":"2024-02-10T06:44:09.215591+0000","last_became_peered":"2024-02-10T06:44:09.215591+0000","last_unstale":"2024-02-10T06:44:45.494258+0000","last_undegraded":"2024-02-10T06:44:45.494258+0000","last_fullsized":"2024-02-10T06:44:45.494258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:11.588475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983135+0000","last_change":"2024-02-10T06:44:07.204319+0000","last_active":"2024-02-10T06:44:44.983135+0000","last_peered":"2024-02-10T06:44:44.983135+0000","last_clean":"2024-02-10T06:44:44.983135+0000","last_became_active":"2024-02-10T06:44:07.203958+0000","last_became_peered":"2024-02-10T06:44:07.203958+0000","last_unstale":"2024-02-10T06:44:44.983135+0000","last_undegraded":"2024-02-10T06:44:44.983135+0000","last_fullsized":"2024-02-10T06:44:44.983135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:09:20.748280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869181+0000","last_change":"2024-02-10T06:44:03.139168+0000","last_active":"2024-02-10T06:44:44.869181+0000","last_peered":"2024-02-10T06:44:44.869181+0000","last_clean":"2024-02-10T06:44:44.869181+0000","last_became_active":"2024-02-10T06:44:03.138865+0000","last_became_peered":"2024-02-10T06:44:03.138865+0000","last_unstale":"2024-02-10T06:44:44.869181+0000","last_undegraded":"2024-02-10T06:44:44.869181+0000","last_fullsized":"2024-02-10T06:44:44.869181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:20.211961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493803+0000","last_change":"2024-02-10T06:44:23.699705+0000","last_active":"2024-02-10T06:44:45.493803+0000","last_peered":"2024-02-10T06:44:45.493803+0000","last_clean":"2024-02-10T06:44:45.493803+0000","last_became_active":"2024-02-10T06:44:23.695308+0000","last_became_peered":"2024-02-10T06:44:23.695308+0000","last_unstale":"2024-02-10T06:44:45.493803+0000","last_undegraded":"2024-02-10T06:44:45.493803+0000","last_fullsized":"2024-02-10T06:44:45.493803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:49:21.319129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869856+0000","last_change":"2024-02-10T06:44:09.207726+0000","last_active":"2024-02-10T06:44:44.869856+0000","last_peered":"2024-02-10T06:44:44.869856+0000","last_clean":"2024-02-10T06:44:44.869856+0000","last_became_active":"2024-02-10T06:44:09.207436+0000","last_became_peered":"2024-02-10T06:44:09.207436+0000","last_unstale":"2024-02-10T06:44:44.869856+0000","last_undegraded":"2024-02-10T06:44:44.869856+0000","last_fullsized":"2024-02-10T06:44:44.869856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:10:17.572855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494367+0000","last_change":"2024-02-10T06:44:07.205472+0000","last_active":"2024-02-10T06:44:45.494367+0000","last_peered":"2024-02-10T06:44:45.494367+0000","last_clean":"2024-02-10T06:44:45.494367+0000","last_became_active":"2024-02-10T06:44:07.205112+0000","last_became_peered":"2024-02-10T06:44:07.205112+0000","last_unstale":"2024-02-10T06:44:45.494367+0000","last_undegraded":"2024-02-10T06:44:45.494367+0000","last_fullsized":"2024-02-10T06:44:45.494367+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:07.273157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919017+0000","last_change":"2024-02-10T06:44:23.671178+0000","last_active":"2024-02-10T06:44:44.919017+0000","last_peered":"2024-02-10T06:44:44.919017+0000","last_clean":"2024-02-10T06:44:44.919017+0000","last_became_active":"2024-02-10T06:44:23.670811+0000","last_became_peered":"2024-02-10T06:44:23.670811+0000","last_unstale":"2024-02-10T06:44:44.919017+0000","last_undegraded":"2024-02-10T06:44:44.919017+0000","last_fullsized":"2024-02-10T06:44:44.919017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:36.914773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493802+0000","last_change":"2024-02-10T06:44:23.604404+0000","last_active":"2024-02-10T06:44:45.493802+0000","last_peered":"2024-02-10T06:44:45.493802+0000","last_clean":"2024-02-10T06:44:45.493802+0000","last_became_active":"2024-02-10T06:44:23.600477+0000","last_became_peered":"2024-02-10T06:44:23.600477+0000","last_unstale":"2024-02-10T06:44:45.493802+0000","last_undegraded":"2024-02-10T06:44:45.493802+0000","last_fullsized":"2024-02-10T06:44:45.493802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:49:16.379676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867220+0000","last_change":"2024-02-10T06:44:09.207023+0000","last_active":"2024-02-10T06:44:44.867220+0000","last_peered":"2024-02-10T06:44:44.867220+0000","last_clean":"2024-02-10T06:44:44.867220+0000","last_became_active":"2024-02-10T06:44:09.206645+0000","last_became_peered":"2024-02-10T06:44:09.206645+0000","last_unstale":"2024-02-10T06:44:44.867220+0000","last_undegraded":"2024-02-10T06:44:44.867220+0000","last_fullsized":"2024-02-10T06:44:44.867220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:21:36.224394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870304+0000","last_change":"2024-02-10T06:44:07.204693+0000","last_active":"2024-02-10T06:44:44.870304+0000","last_peered":"2024-02-10T06:44:44.870304+0000","last_clean":"2024-02-10T06:44:44.870304+0000","last_became_active":"2024-02-10T06:44:07.204377+0000","last_became_peered":"2024-02-10T06:44:07.204377+0000","last_unstale":"2024-02-10T06:44:44.870304+0000","last_undegraded":"2024-02-10T06:44:44.870304+0000","last_fullsized":"2024-02-10T06:44:44.870304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:09.733042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495181+0000","last_change":"2024-02-10T06:44:03.135904+0000","last_active":"2024-02-10T06:44:45.495181+0000","last_peered":"2024-02-10T06:44:45.495181+0000","last_clean":"2024-02-10T06:44:45.495181+0000","last_became_active":"2024-02-10T06:44:03.135624+0000","last_became_peered":"2024-02-10T06:44:03.135624+0000","last_unstale":"2024-02-10T06:44:45.495181+0000","last_undegraded":"2024-02-10T06:44:45.495181+0000","last_fullsized":"2024-02-10T06:44:45.495181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:34:24.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493640+0000","last_change":"2024-02-10T06:44:05.190339+0000","last_active":"2024-02-10T06:44:45.493640+0000","last_peered":"2024-02-10T06:44:45.493640+0000","last_clean":"2024-02-10T06:44:45.493640+0000","last_became_active":"2024-02-10T06:44:05.190029+0000","last_became_peered":"2024-02-10T06:44:05.190029+0000","last_unstale":"2024-02-10T06:44:45.493640+0000","last_undegraded":"2024-02-10T06:44:45.493640+0000","last_fullsized":"2024-02-10T06:44:45.493640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:19:24.957844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869422+0000","last_change":"2024-02-10T06:44:09.207570+0000","last_active":"2024-02-10T06:44:44.869422+0000","last_peered":"2024-02-10T06:44:44.869422+0000","last_clean":"2024-02-10T06:44:44.869422+0000","last_became_active":"2024-02-10T06:44:09.207300+0000","last_became_peered":"2024-02-10T06:44:09.207300+0000","last_unstale":"2024-02-10T06:44:44.869422+0000","last_undegraded":"2024-02-10T06:44:44.869422+0000","last_fullsized":"2024-02-10T06:44:44.869422+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:00:02.470714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493968+0000","last_change":"2024-02-10T06:44:07.193765+0000","last_active":"2024-02-10T06:44:45.493968+0000","last_peered":"2024-02-10T06:44:45.493968+0000","last_clean":"2024-02-10T06:44:45.493968+0000","last_became_active":"2024-02-10T06:44:07.193462+0000","last_became_peered":"2024-02-10T06:44:07.193462+0000","last_unstale":"2024-02-10T06:44:45.493968+0000","last_undegraded":"2024-02-10T06:44:45.493968+0000","last_fullsized":"2024-02-10T06:44:45.493968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:06:52.310075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868613+0000","last_change":"2024-02-10T06:44:03.133714+0000","last_active":"2024-02-10T06:44:44.868613+0000","last_peered":"2024-02-10T06:44:44.868613+0000","last_clean":"2024-02-10T06:44:44.868613+0000","last_became_active":"2024-02-10T06:44:03.133395+0000","last_became_peered":"2024-02-10T06:44:03.133395+0000","last_unstale":"2024-02-10T06:44:44.868613+0000","last_undegraded":"2024-02-10T06:44:44.868613+0000","last_fullsized":"2024-02-10T06:44:44.868613+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:18:18.057639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868947+0000","last_change":"2024-02-10T06:44:05.198626+0000","last_active":"2024-02-10T06:44:44.868947+0000","last_peered":"2024-02-10T06:44:44.868947+0000","last_clean":"2024-02-10T06:44:44.868947+0000","last_became_active":"2024-02-10T06:44:05.198039+0000","last_became_peered":"2024-02-10T06:44:05.198039+0000","last_unstale":"2024-02-10T06:44:44.868947+0000","last_undegraded":"2024-02-10T06:44:44.868947+0000","last_fullsized":"2024-02-10T06:44:44.868947+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:52:02.512152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495646+0000","last_change":"2024-02-10T06:44:09.210492+0000","last_active":"2024-02-10T06:44:45.495646+0000","last_peered":"2024-02-10T06:44:45.495646+0000","last_clean":"2024-02-10T06:44:45.495646+0000","last_became_active":"2024-02-10T06:44:09.210162+0000","last_became_peered":"2024-02-10T06:44:09.210162+0000","last_unstale":"2024-02-10T06:44:45.495646+0000","last_undegraded":"2024-02-10T06:44:45.495646+0000","last_fullsized":"2024-02-10T06:44:45.495646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:02:02.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984684+0000","last_change":"2024-02-10T06:44:07.206349+0000","last_active":"2024-02-10T06:44:44.984684+0000","last_peered":"2024-02-10T06:44:44.984684+0000","last_clean":"2024-02-10T06:44:44.984684+0000","last_became_active":"2024-02-10T06:44:07.205996+0000","last_became_peered":"2024-02-10T06:44:07.205996+0000","last_unstale":"2024-02-10T06:44:44.984684+0000","last_undegraded":"2024-02-10T06:44:44.984684+0000","last_fullsized":"2024-02-10T06:44:44.984684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:40:27.716618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496018+0000","last_change":"2024-02-10T06:44:03.133745+0000","last_active":"2024-02-10T06:44:45.496018+0000","last_peered":"2024-02-10T06:44:45.496018+0000","last_clean":"2024-02-10T06:44:45.496018+0000","last_became_active":"2024-02-10T06:44:03.133024+0000","last_became_peered":"2024-02-10T06:44:03.133024+0000","last_unstale":"2024-02-10T06:44:45.496018+0000","last_undegraded":"2024-02-10T06:44:45.496018+0000","last_fullsized":"2024-02-10T06:44:45.496018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:50:02.399559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919354+0000","last_change":"2024-02-10T06:44:05.196604+0000","last_active":"2024-02-10T06:44:44.919354+0000","last_peered":"2024-02-10T06:44:44.919354+0000","last_clean":"2024-02-10T06:44:44.919354+0000","last_became_active":"2024-02-10T06:44:05.196319+0000","last_became_peered":"2024-02-10T06:44:05.196319+0000","last_unstale":"2024-02-10T06:44:44.919354+0000","last_undegraded":"2024-02-10T06:44:44.919354+0000","last_fullsized":"2024-02-10T06:44:44.919354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:20:40.973169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984163+0000","last_change":"2024-02-10T06:44:09.212123+0000","last_active":"2024-02-10T06:44:44.984163+0000","last_peered":"2024-02-10T06:44:44.984163+0000","last_clean":"2024-02-10T06:44:44.984163+0000","last_became_active":"2024-02-10T06:44:09.211826+0000","last_became_peered":"2024-02-10T06:44:09.211826+0000","last_unstale":"2024-02-10T06:44:44.984163+0000","last_undegraded":"2024-02-10T06:44:44.984163+0000","last_fullsized":"2024-02-10T06:44:44.984163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:11:48.697285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868702+0000","last_change":"2024-02-10T06:44:07.197269+0000","last_active":"2024-02-10T06:44:44.868702+0000","last_peered":"2024-02-10T06:44:44.868702+0000","last_clean":"2024-02-10T06:44:44.868702+0000","last_became_active":"2024-02-10T06:44:07.196949+0000","last_became_peered":"2024-02-10T06:44:07.196949+0000","last_unstale":"2024-02-10T06:44:44.868702+0000","last_undegraded":"2024-02-10T06:44:44.868702+0000","last_fullsized":"2024-02-10T06:44:44.868702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:16:51.826635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"60'1","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868298+0000","last_change":"2024-02-10T06:44:03.134052+0000","last_active":"2024-02-10T06:44:44.868298+0000","last_peered":"2024-02-10T06:44:44.868298+0000","last_clean":"2024-02-10T06:44:44.868298+0000","last_became_active":"2024-02-10T06:44:03.133505+0000","last_became_peered":"2024-02-10T06:44:03.133505+0000","last_unstale":"2024-02-10T06:44:44.868298+0000","last_undegraded":"2024-02-10T06:44:44.868298+0000","last_fullsized":"2024-02-10T06:44:44.868298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:09.097584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496171+0000","last_change":"2024-02-10T06:44:05.197133+0000","last_active":"2024-02-10T06:44:45.496171+0000","last_peered":"2024-02-10T06:44:45.496171+0000","last_clean":"2024-02-10T06:44:45.496171+0000","last_became_active":"2024-02-10T06:44:05.196816+0000","last_became_peered":"2024-02-10T06:44:05.196816+0000","last_unstale":"2024-02-10T06:44:45.496171+0000","last_undegraded":"2024-02-10T06:44:45.496171+0000","last_fullsized":"2024-02-10T06:44:45.496171+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:53:30.745628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495515+0000","last_change":"2024-02-10T06:44:09.214454+0000","last_active":"2024-02-10T06:44:45.495515+0000","last_peered":"2024-02-10T06:44:45.495515+0000","last_clean":"2024-02-10T06:44:45.495515+0000","last_became_active":"2024-02-10T06:44:09.214129+0000","last_became_peered":"2024-02-10T06:44:09.214129+0000","last_unstale":"2024-02-10T06:44:45.495515+0000","last_undegraded":"2024-02-10T06:44:45.495515+0000","last_fullsized":"2024-02-10T06:44:45.495515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:03:27.408367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"67'8","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194726+0000","last_change":"2024-02-10T06:44:07.201769+0000","last_active":"2024-02-10T06:45:23.194726+0000","last_peered":"2024-02-10T06:45:23.194726+0000","last_clean":"2024-02-10T06:45:23.194726+0000","last_became_active":"2024-02-10T06:44:07.201482+0000","last_became_peered":"2024-02-10T06:44:07.201482+0000","last_unstale":"2024-02-10T06:45:23.194726+0000","last_undegraded":"2024-02-10T06:45:23.194726+0000","last_fullsized":"2024-02-10T06:45:23.194726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:14:39.019619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869815+0000","last_change":"2024-02-10T06:44:03.130230+0000","last_active":"2024-02-10T06:44:44.869815+0000","last_peered":"2024-02-10T06:44:44.869815+0000","last_clean":"2024-02-10T06:44:44.869815+0000","last_became_active":"2024-02-10T06:44:03.129949+0000","last_became_peered":"2024-02-10T06:44:03.129949+0000","last_unstale":"2024-02-10T06:44:44.869815+0000","last_undegraded":"2024-02-10T06:44:44.869815+0000","last_fullsized":"2024-02-10T06:44:44.869815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:00:31.285542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868583+0000","last_change":"2024-02-10T06:44:05.332344+0000","last_active":"2024-02-10T06:44:44.868583+0000","last_peered":"2024-02-10T06:44:44.868583+0000","last_clean":"2024-02-10T06:44:44.868583+0000","last_became_active":"2024-02-10T06:44:05.332099+0000","last_became_peered":"2024-02-10T06:44:05.332099+0000","last_unstale":"2024-02-10T06:44:44.868583+0000","last_undegraded":"2024-02-10T06:44:44.868583+0000","last_fullsized":"2024-02-10T06:44:44.868583+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:34:11.834836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868887+0000","last_change":"2024-02-10T06:44:09.201694+0000","last_active":"2024-02-10T06:44:44.868887+0000","last_peered":"2024-02-10T06:44:44.868887+0000","last_clean":"2024-02-10T06:44:44.868887+0000","last_became_active":"2024-02-10T06:44:09.201290+0000","last_became_peered":"2024-02-10T06:44:09.201290+0000","last_unstale":"2024-02-10T06:44:44.868887+0000","last_undegraded":"2024-02-10T06:44:44.868887+0000","last_fullsized":"2024-02-10T06:44:44.868887+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:46:55.440561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494601+0000","last_change":"2024-02-10T06:44:07.203652+0000","last_active":"2024-02-10T06:44:45.494601+0000","last_peered":"2024-02-10T06:44:45.494601+0000","last_clean":"2024-02-10T06:44:45.494601+0000","last_became_active":"2024-02-10T06:44:07.203355+0000","last_became_peered":"2024-02-10T06:44:07.203355+0000","last_unstale":"2024-02-10T06:44:45.494601+0000","last_undegraded":"2024-02-10T06:44:45.494601+0000","last_fullsized":"2024-02-10T06:44:45.494601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:30:15.283761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496572+0000","last_change":"2024-02-10T06:44:03.131244+0000","last_active":"2024-02-10T06:44:45.496572+0000","last_peered":"2024-02-10T06:44:45.496572+0000","last_clean":"2024-02-10T06:44:45.496572+0000","last_became_active":"2024-02-10T06:44:03.130884+0000","last_became_peered":"2024-02-10T06:44:03.130884+0000","last_unstale":"2024-02-10T06:44:45.496572+0000","last_undegraded":"2024-02-10T06:44:45.496572+0000","last_fullsized":"2024-02-10T06:44:45.496572+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:42:21.848411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868633+0000","last_change":"2024-02-10T06:44:05.332063+0000","last_active":"2024-02-10T06:44:44.868633+0000","last_peered":"2024-02-10T06:44:44.868633+0000","last_clean":"2024-02-10T06:44:44.868633+0000","last_became_active":"2024-02-10T06:44:05.331667+0000","last_became_peered":"2024-02-10T06:44:05.331667+0000","last_unstale":"2024-02-10T06:44:44.868633+0000","last_undegraded":"2024-02-10T06:44:44.868633+0000","last_fullsized":"2024-02-10T06:44:44.868633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:44:09.459671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494138+0000","last_change":"2024-02-10T06:44:09.213087+0000","last_active":"2024-02-10T06:44:45.494138+0000","last_peered":"2024-02-10T06:44:45.494138+0000","last_clean":"2024-02-10T06:44:45.494138+0000","last_became_active":"2024-02-10T06:44:09.212681+0000","last_became_peered":"2024-02-10T06:44:09.212681+0000","last_unstale":"2024-02-10T06:44:45.494138+0000","last_undegraded":"2024-02-10T06:44:45.494138+0000","last_fullsized":"2024-02-10T06:44:45.494138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:57:50.506314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494972+0000","last_change":"2024-02-10T06:44:07.200774+0000","last_active":"2024-02-10T06:44:45.494972+0000","last_peered":"2024-02-10T06:44:45.494972+0000","last_clean":"2024-02-10T06:44:45.494972+0000","last_became_active":"2024-02-10T06:44:07.200506+0000","last_became_peered":"2024-02-10T06:44:07.200506+0000","last_unstale":"2024-02-10T06:44:45.494972+0000","last_undegraded":"2024-02-10T06:44:45.494972+0000","last_fullsized":"2024-02-10T06:44:45.494972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:07:59.365259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870817+0000","last_change":"2024-02-10T06:44:03.136894+0000","last_active":"2024-02-10T06:44:44.870817+0000","last_peered":"2024-02-10T06:44:44.870817+0000","last_clean":"2024-02-10T06:44:44.870817+0000","last_became_active":"2024-02-10T06:44:03.136599+0000","last_became_peered":"2024-02-10T06:44:03.136599+0000","last_unstale":"2024-02-10T06:44:44.870817+0000","last_undegraded":"2024-02-10T06:44:44.870817+0000","last_fullsized":"2024-02-10T06:44:44.870817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:53:00.098332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868025+0000","last_change":"2024-02-10T06:44:05.197443+0000","last_active":"2024-02-10T06:44:44.868025+0000","last_peered":"2024-02-10T06:44:44.868025+0000","last_clean":"2024-02-10T06:44:44.868025+0000","last_became_active":"2024-02-10T06:44:05.197025+0000","last_became_peered":"2024-02-10T06:44:05.197025+0000","last_unstale":"2024-02-10T06:44:44.868025+0000","last_undegraded":"2024-02-10T06:44:44.868025+0000","last_fullsized":"2024-02-10T06:44:44.868025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:46:36.237275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868051+0000","last_change":"2024-02-10T06:44:09.205073+0000","last_active":"2024-02-10T06:44:44.868051+0000","last_peered":"2024-02-10T06:44:44.868051+0000","last_clean":"2024-02-10T06:44:44.868051+0000","last_became_active":"2024-02-10T06:44:09.204758+0000","last_became_peered":"2024-02-10T06:44:09.204758+0000","last_unstale":"2024-02-10T06:44:44.868051+0000","last_undegraded":"2024-02-10T06:44:44.868051+0000","last_fullsized":"2024-02-10T06:44:44.868051+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:04:18.216542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983702+0000","last_change":"2024-02-10T06:44:07.203090+0000","last_active":"2024-02-10T06:44:44.983702+0000","last_peered":"2024-02-10T06:44:44.983702+0000","last_clean":"2024-02-10T06:44:44.983702+0000","last_became_active":"2024-02-10T06:44:07.202791+0000","last_became_peered":"2024-02-10T06:44:07.202791+0000","last_unstale":"2024-02-10T06:44:44.983702+0000","last_undegraded":"2024-02-10T06:44:44.983702+0000","last_fullsized":"2024-02-10T06:44:44.983702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:37.600065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984329+0000","last_change":"2024-02-10T06:44:03.139011+0000","last_active":"2024-02-10T06:44:44.984329+0000","last_peered":"2024-02-10T06:44:44.984329+0000","last_clean":"2024-02-10T06:44:44.984329+0000","last_became_active":"2024-02-10T06:44:03.138713+0000","last_became_peered":"2024-02-10T06:44:03.138713+0000","last_unstale":"2024-02-10T06:44:44.984329+0000","last_undegraded":"2024-02-10T06:44:44.984329+0000","last_fullsized":"2024-02-10T06:44:44.984329+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:53.842077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493374+0000","last_change":"2024-02-10T06:44:05.198245+0000","last_active":"2024-02-10T06:44:45.493374+0000","last_peered":"2024-02-10T06:44:45.493374+0000","last_clean":"2024-02-10T06:44:45.493374+0000","last_became_active":"2024-02-10T06:44:05.197240+0000","last_became_peered":"2024-02-10T06:44:05.197240+0000","last_unstale":"2024-02-10T06:44:45.493374+0000","last_undegraded":"2024-02-10T06:44:45.493374+0000","last_fullsized":"2024-02-10T06:44:45.493374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:06:41.694172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.866813+0000","last_change":"2024-02-10T06:44:09.205396+0000","last_active":"2024-02-10T06:44:44.866813+0000","last_peered":"2024-02-10T06:44:44.866813+0000","last_clean":"2024-02-10T06:44:44.866813+0000","last_became_active":"2024-02-10T06:44:09.205124+0000","last_became_peered":"2024-02-10T06:44:09.205124+0000","last_unstale":"2024-02-10T06:44:44.866813+0000","last_undegraded":"2024-02-10T06:44:44.866813+0000","last_fullsized":"2024-02-10T06:44:44.866813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:12:27.596628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868701+0000","last_change":"2024-02-10T06:44:07.197256+0000","last_active":"2024-02-10T06:44:44.868701+0000","last_peered":"2024-02-10T06:44:44.868701+0000","last_clean":"2024-02-10T06:44:44.868701+0000","last_became_active":"2024-02-10T06:44:07.196972+0000","last_became_peered":"2024-02-10T06:44:07.196972+0000","last_unstale":"2024-02-10T06:44:44.868701+0000","last_undegraded":"2024-02-10T06:44:44.868701+0000","last_fullsized":"2024-02-10T06:44:44.868701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:08:02.938962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496327+0000","last_change":"2024-02-10T06:44:03.131992+0000","last_active":"2024-02-10T06:44:45.496327+0000","last_peered":"2024-02-10T06:44:45.496327+0000","last_clean":"2024-02-10T06:44:45.496327+0000","last_became_active":"2024-02-10T06:44:03.131666+0000","last_became_peered":"2024-02-10T06:44:03.131666+0000","last_unstale":"2024-02-10T06:44:45.496327+0000","last_undegraded":"2024-02-10T06:44:45.496327+0000","last_fullsized":"2024-02-10T06:44:45.496327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:39:04.101751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983381+0000","last_change":"2024-02-10T06:44:05.195591+0000","last_active":"2024-02-10T06:44:44.983381+0000","last_peered":"2024-02-10T06:44:44.983381+0000","last_clean":"2024-02-10T06:44:44.983381+0000","last_became_active":"2024-02-10T06:44:05.195292+0000","last_became_peered":"2024-02-10T06:44:05.195292+0000","last_unstale":"2024-02-10T06:44:44.983381+0000","last_undegraded":"2024-02-10T06:44:44.983381+0000","last_fullsized":"2024-02-10T06:44:44.983381+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:45:21.870785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495485+0000","last_change":"2024-02-10T06:44:09.208960+0000","last_active":"2024-02-10T06:44:45.495485+0000","last_peered":"2024-02-10T06:44:45.495485+0000","last_clean":"2024-02-10T06:44:45.495485+0000","last_became_active":"2024-02-10T06:44:09.208668+0000","last_became_peered":"2024-02-10T06:44:09.208668+0000","last_unstale":"2024-02-10T06:44:45.495485+0000","last_undegraded":"2024-02-10T06:44:45.495485+0000","last_fullsized":"2024-02-10T06:44:45.495485+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:53:49.029054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194573+0000","last_change":"2024-02-10T06:44:07.199098+0000","last_active":"2024-02-10T06:45:23.194573+0000","last_peered":"2024-02-10T06:45:23.194573+0000","last_clean":"2024-02-10T06:45:23.194573+0000","last_became_active":"2024-02-10T06:44:07.198779+0000","last_became_peered":"2024-02-10T06:44:07.198779+0000","last_unstale":"2024-02-10T06:45:23.194573+0000","last_undegraded":"2024-02-10T06:45:23.194573+0000","last_fullsized":"2024-02-10T06:45:23.194573+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:06:04.122479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"60'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867368+0000","last_change":"2024-02-10T06:44:03.139192+0000","last_active":"2024-02-10T06:44:44.867368+0000","last_peered":"2024-02-10T06:44:44.867368+0000","last_clean":"2024-02-10T06:44:44.867368+0000","last_became_active":"2024-02-10T06:44:03.138301+0000","last_became_peered":"2024-02-10T06:44:03.138301+0000","last_unstale":"2024-02-10T06:44:44.867368+0000","last_undegraded":"2024-02-10T06:44:44.867368+0000","last_fullsized":"2024-02-10T06:44:44.867368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:08:28.173573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870838+0000","last_change":"2024-02-10T06:44:05.332054+0000","last_active":"2024-02-10T06:44:44.870838+0000","last_peered":"2024-02-10T06:44:44.870838+0000","last_clean":"2024-02-10T06:44:44.870838+0000","last_became_active":"2024-02-10T06:44:05.331722+0000","last_became_peered":"2024-02-10T06:44:05.331722+0000","last_unstale":"2024-02-10T06:44:44.870838+0000","last_undegraded":"2024-02-10T06:44:44.870838+0000","last_fullsized":"2024-02-10T06:44:44.870838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:57:06.187835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868159+0000","last_change":"2024-02-10T06:44:09.206729+0000","last_active":"2024-02-10T06:44:44.868159+0000","last_peered":"2024-02-10T06:44:44.868159+0000","last_clean":"2024-02-10T06:44:44.868159+0000","last_became_active":"2024-02-10T06:44:09.206375+0000","last_became_peered":"2024-02-10T06:44:09.206375+0000","last_unstale":"2024-02-10T06:44:44.868159+0000","last_undegraded":"2024-02-10T06:44:44.868159+0000","last_fullsized":"2024-02-10T06:44:44.868159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:11:32.326758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194732+0000","last_change":"2024-02-10T06:44:07.201557+0000","last_active":"2024-02-10T06:45:23.194732+0000","last_peered":"2024-02-10T06:45:23.194732+0000","last_clean":"2024-02-10T06:45:23.194732+0000","last_became_active":"2024-02-10T06:44:07.201263+0000","last_became_peered":"2024-02-10T06:44:07.201263+0000","last_unstale":"2024-02-10T06:45:23.194732+0000","last_undegraded":"2024-02-10T06:45:23.194732+0000","last_fullsized":"2024-02-10T06:45:23.194732+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:31:44.221235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869986+0000","last_change":"2024-02-10T06:44:03.131413+0000","last_active":"2024-02-10T06:44:44.869986+0000","last_peered":"2024-02-10T06:44:44.869986+0000","last_clean":"2024-02-10T06:44:44.869986+0000","last_became_active":"2024-02-10T06:44:03.131137+0000","last_became_peered":"2024-02-10T06:44:03.131137+0000","last_unstale":"2024-02-10T06:44:44.869986+0000","last_undegraded":"2024-02-10T06:44:44.869986+0000","last_fullsized":"2024-02-10T06:44:44.869986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:08:35.332808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868460+0000","last_change":"2024-02-10T06:44:05.332734+0000","last_active":"2024-02-10T06:44:44.868460+0000","last_peered":"2024-02-10T06:44:44.868460+0000","last_clean":"2024-02-10T06:44:44.868460+0000","last_became_active":"2024-02-10T06:44:05.332217+0000","last_became_peered":"2024-02-10T06:44:05.332217+0000","last_unstale":"2024-02-10T06:44:44.868460+0000","last_undegraded":"2024-02-10T06:44:44.868460+0000","last_fullsized":"2024-02-10T06:44:44.868460+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:15:07.935178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495713+0000","last_change":"2024-02-10T06:44:09.214695+0000","last_active":"2024-02-10T06:44:45.495713+0000","last_peered":"2024-02-10T06:44:45.495713+0000","last_clean":"2024-02-10T06:44:45.495713+0000","last_became_active":"2024-02-10T06:44:09.214422+0000","last_became_peered":"2024-02-10T06:44:09.214422+0000","last_unstale":"2024-02-10T06:44:45.495713+0000","last_undegraded":"2024-02-10T06:44:45.495713+0000","last_fullsized":"2024-02-10T06:44:45.495713+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:45:09.044749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870735+0000","last_change":"2024-02-10T06:44:07.206445+0000","last_active":"2024-02-10T06:44:44.870735+0000","last_peered":"2024-02-10T06:44:44.870735+0000","last_clean":"2024-02-10T06:44:44.870735+0000","last_became_active":"2024-02-10T06:44:07.206157+0000","last_became_peered":"2024-02-10T06:44:07.206157+0000","last_unstale":"2024-02-10T06:44:44.870735+0000","last_undegraded":"2024-02-10T06:44:44.870735+0000","last_fullsized":"2024-02-10T06:44:44.870735+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:12:44.835653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496485+0000","last_change":"2024-02-10T06:44:03.132818+0000","last_active":"2024-02-10T06:44:45.496485+0000","last_peered":"2024-02-10T06:44:45.496485+0000","last_clean":"2024-02-10T06:44:45.496485+0000","last_became_active":"2024-02-10T06:44:03.132358+0000","last_became_peered":"2024-02-10T06:44:03.132358+0000","last_unstale":"2024-02-10T06:44:45.496485+0000","last_undegraded":"2024-02-10T06:44:45.496485+0000","last_fullsized":"2024-02-10T06:44:45.496485+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:51:30.613464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869404+0000","last_change":"2024-02-10T06:44:05.331978+0000","last_active":"2024-02-10T06:44:44.869404+0000","last_peered":"2024-02-10T06:44:44.869404+0000","last_clean":"2024-02-10T06:44:44.869404+0000","last_became_active":"2024-02-10T06:44:05.331646+0000","last_became_peered":"2024-02-10T06:44:05.331646+0000","last_unstale":"2024-02-10T06:44:44.869404+0000","last_undegraded":"2024-02-10T06:44:44.869404+0000","last_fullsized":"2024-02-10T06:44:44.869404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:59:37.407166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919060+0000","last_change":"2024-02-10T06:44:09.211639+0000","last_active":"2024-02-10T06:44:44.919060+0000","last_peered":"2024-02-10T06:44:44.919060+0000","last_clean":"2024-02-10T06:44:44.919060+0000","last_became_active":"2024-02-10T06:44:09.211286+0000","last_became_peered":"2024-02-10T06:44:09.211286+0000","last_unstale":"2024-02-10T06:44:44.919060+0000","last_undegraded":"2024-02-10T06:44:44.919060+0000","last_fullsized":"2024-02-10T06:44:44.919060+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:09:41.661125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869509+0000","last_change":"2024-02-10T06:44:07.198398+0000","last_active":"2024-02-10T06:44:44.869509+0000","last_peered":"2024-02-10T06:44:44.869509+0000","last_clean":"2024-02-10T06:44:44.869509+0000","last_became_active":"2024-02-10T06:44:07.198073+0000","last_became_peered":"2024-02-10T06:44:07.198073+0000","last_unstale":"2024-02-10T06:44:44.869509+0000","last_undegraded":"2024-02-10T06:44:44.869509+0000","last_fullsized":"2024-02-10T06:44:44.869509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:19:16.672133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918812+0000","last_change":"2024-02-10T06:44:03.142957+0000","last_active":"2024-02-10T06:44:44.918812+0000","last_peered":"2024-02-10T06:44:44.918812+0000","last_clean":"2024-02-10T06:44:44.918812+0000","last_became_active":"2024-02-10T06:44:03.142675+0000","last_became_peered":"2024-02-10T06:44:03.142675+0000","last_unstale":"2024-02-10T06:44:44.918812+0000","last_undegraded":"2024-02-10T06:44:44.918812+0000","last_fullsized":"2024-02-10T06:44:44.918812+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:06:47.377948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868355+0000","last_change":"2024-02-10T06:44:05.331521+0000","last_active":"2024-02-10T06:44:44.868355+0000","last_peered":"2024-02-10T06:44:44.868355+0000","last_clean":"2024-02-10T06:44:44.868355+0000","last_became_active":"2024-02-10T06:44:05.331221+0000","last_became_peered":"2024-02-10T06:44:05.331221+0000","last_unstale":"2024-02-10T06:44:44.868355+0000","last_undegraded":"2024-02-10T06:44:44.868355+0000","last_fullsized":"2024-02-10T06:44:44.868355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:02:31.669237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868788+0000","last_change":"2024-02-10T06:44:09.206739+0000","last_active":"2024-02-10T06:44:44.868788+0000","last_peered":"2024-02-10T06:44:44.868788+0000","last_clean":"2024-02-10T06:44:44.868788+0000","last_became_active":"2024-02-10T06:44:09.206429+0000","last_became_peered":"2024-02-10T06:44:09.206429+0000","last_unstale":"2024-02-10T06:44:44.868788+0000","last_undegraded":"2024-02-10T06:44:44.868788+0000","last_fullsized":"2024-02-10T06:44:44.868788+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:51:42.777986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495265+0000","last_change":"2024-02-10T06:44:03.137085+0000","last_active":"2024-02-10T06:44:45.495265+0000","last_peered":"2024-02-10T06:44:45.495265+0000","last_clean":"2024-02-10T06:44:45.495265+0000","last_became_active":"2024-02-10T06:44:03.136782+0000","last_became_peered":"2024-02-10T06:44:03.136782+0000","last_unstale":"2024-02-10T06:44:45.495265+0000","last_undegraded":"2024-02-10T06:44:45.495265+0000","last_fullsized":"2024-02-10T06:44:45.495265+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:19:33.094592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":52,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493134+0000","last_change":"2024-02-10T06:44:05.196500+0000","last_active":"2024-02-10T06:44:45.493134+0000","last_peered":"2024-02-10T06:44:45.493134+0000","last_clean":"2024-02-10T06:44:45.493134+0000","last_became_active":"2024-02-10T06:44:05.196234+0000","last_became_peered":"2024-02-10T06:44:05.196234+0000","last_unstale":"2024-02-10T06:44:45.493134+0000","last_undegraded":"2024-02-10T06:44:45.493134+0000","last_fullsized":"2024-02-10T06:44:45.493134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:17:33.731568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918808+0000","last_change":"2024-02-10T06:44:07.202717+0000","last_active":"2024-02-10T06:44:44.918808+0000","last_peered":"2024-02-10T06:44:44.918808+0000","last_clean":"2024-02-10T06:44:44.918808+0000","last_became_active":"2024-02-10T06:44:07.202403+0000","last_became_peered":"2024-02-10T06:44:07.202403+0000","last_unstale":"2024-02-10T06:44:44.918808+0000","last_undegraded":"2024-02-10T06:44:44.918808+0000","last_fullsized":"2024-02-10T06:44:44.918808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:45:16.320601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.866813+0000","last_change":"2024-02-10T06:44:09.200057+0000","last_active":"2024-02-10T06:44:44.866813+0000","last_peered":"2024-02-10T06:44:44.866813+0000","last_clean":"2024-02-10T06:44:44.866813+0000","last_became_active":"2024-02-10T06:44:09.199734+0000","last_became_peered":"2024-02-10T06:44:09.199734+0000","last_unstale":"2024-02-10T06:44:44.866813+0000","last_undegraded":"2024-02-10T06:44:44.866813+0000","last_fullsized":"2024-02-10T06:44:44.866813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:57:10.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984070+0000","last_change":"2024-02-10T06:44:03.139457+0000","last_active":"2024-02-10T06:44:44.984070+0000","last_peered":"2024-02-10T06:44:44.984070+0000","last_clean":"2024-02-10T06:44:44.984070+0000","last_became_active":"2024-02-10T06:44:03.139178+0000","last_became_peered":"2024-02-10T06:44:03.139178+0000","last_unstale":"2024-02-10T06:44:44.984070+0000","last_undegraded":"2024-02-10T06:44:44.984070+0000","last_fullsized":"2024-02-10T06:44:44.984070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:41:42.617087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983450+0000","last_change":"2024-02-10T06:44:05.199342+0000","last_active":"2024-02-10T06:44:44.983450+0000","last_peered":"2024-02-10T06:44:44.983450+0000","last_clean":"2024-02-10T06:44:44.983450+0000","last_became_active":"2024-02-10T06:44:05.198930+0000","last_became_peered":"2024-02-10T06:44:05.198930+0000","last_unstale":"2024-02-10T06:44:44.983450+0000","last_undegraded":"2024-02-10T06:44:44.983450+0000","last_fullsized":"2024-02-10T06:44:44.983450+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:39:32.171612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194477+0000","last_change":"2024-02-10T06:44:07.204235+0000","last_active":"2024-02-10T06:45:23.194477+0000","last_peered":"2024-02-10T06:45:23.194477+0000","last_clean":"2024-02-10T06:45:23.194477+0000","last_became_active":"2024-02-10T06:44:07.203514+0000","last_became_peered":"2024-02-10T06:44:07.203514+0000","last_unstale":"2024-02-10T06:45:23.194477+0000","last_undegraded":"2024-02-10T06:45:23.194477+0000","last_fullsized":"2024-02-10T06:45:23.194477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:57:53.155613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869198+0000","last_change":"2024-02-10T06:44:09.210017+0000","last_active":"2024-02-10T06:44:44.869198+0000","last_peered":"2024-02-10T06:44:44.869198+0000","last_clean":"2024-02-10T06:44:44.869198+0000","last_became_active":"2024-02-10T06:44:09.209705+0000","last_became_peered":"2024-02-10T06:44:09.209705+0000","last_unstale":"2024-02-10T06:44:44.869198+0000","last_undegraded":"2024-02-10T06:44:44.869198+0000","last_fullsized":"2024-02-10T06:44:44.869198+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:29.690364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870077+0000","last_change":"2024-02-10T06:44:03.137411+0000","last_active":"2024-02-10T06:44:44.870077+0000","last_peered":"2024-02-10T06:44:44.870077+0000","last_clean":"2024-02-10T06:44:44.870077+0000","last_became_active":"2024-02-10T06:44:03.137124+0000","last_became_peered":"2024-02-10T06:44:03.137124+0000","last_unstale":"2024-02-10T06:44:44.870077+0000","last_undegraded":"2024-02-10T06:44:44.870077+0000","last_fullsized":"2024-02-10T06:44:44.870077+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:28:36.069474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"67'28","reported_seq":95,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194483+0000","last_change":"2024-02-10T06:44:05.196676+0000","last_active":"2024-02-10T06:45:23.194483+0000","last_peered":"2024-02-10T06:45:23.194483+0000","last_clean":"2024-02-10T06:45:23.194483+0000","last_became_active":"2024-02-10T06:44:05.196381+0000","last_became_peered":"2024-02-10T06:44:05.196381+0000","last_unstale":"2024-02-10T06:45:23.194483+0000","last_undegraded":"2024-02-10T06:45:23.194483+0000","last_fullsized":"2024-02-10T06:45:23.194483+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:21:07.356318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867744+0000","last_change":"2024-02-10T06:44:07.203961+0000","last_active":"2024-02-10T06:44:44.867744+0000","last_peered":"2024-02-10T06:44:44.867744+0000","last_clean":"2024-02-10T06:44:44.867744+0000","last_became_active":"2024-02-10T06:44:07.203125+0000","last_became_peered":"2024-02-10T06:44:07.203125+0000","last_unstale":"2024-02-10T06:44:44.867744+0000","last_undegraded":"2024-02-10T06:44:44.867744+0000","last_fullsized":"2024-02-10T06:44:44.867744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:30:35.632812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495813+0000","last_change":"2024-02-10T06:44:09.215807+0000","last_active":"2024-02-10T06:44:45.495813+0000","last_peered":"2024-02-10T06:44:45.495813+0000","last_clean":"2024-02-10T06:44:45.495813+0000","last_became_active":"2024-02-10T06:44:09.215552+0000","last_became_peered":"2024-02-10T06:44:09.215552+0000","last_unstale":"2024-02-10T06:44:45.495813+0000","last_undegraded":"2024-02-10T06:44:45.495813+0000","last_fullsized":"2024-02-10T06:44:45.495813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:20:30.119417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"67'5","reported_seq":125,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:29.567549+0000","last_change":"2024-02-10T06:44:07.200823+0000","last_active":"2024-02-10T06:45:29.567549+0000","last_peered":"2024-02-10T06:45:29.567549+0000","last_clean":"2024-02-10T06:45:29.567549+0000","last_became_active":"2024-02-10T06:44:03.137370+0000","last_became_peered":"2024-02-10T06:44:03.137370+0000","last_unstale":"2024-02-10T06:45:29.567549+0000","last_undegraded":"2024-02-10T06:45:29.567549+0000","last_fullsized":"2024-02-10T06:45:29.567549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:20:00.259539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049614910000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":82,"num_read_kb":77,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918786+0000","last_change":"2024-02-10T06:44:03.140681+0000","last_active":"2024-02-10T06:44:44.918786+0000","last_peered":"2024-02-10T06:44:44.918786+0000","last_clean":"2024-02-10T06:44:44.918786+0000","last_became_active":"2024-02-10T06:44:03.140442+0000","last_became_peered":"2024-02-10T06:44:03.140442+0000","last_unstale":"2024-02-10T06:44:44.918786+0000","last_undegraded":"2024-02-10T06:44:44.918786+0000","last_fullsized":"2024-02-10T06:44:44.918786+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:03:11.632597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984801+0000","last_change":"2024-02-10T06:44:05.197477+0000","last_active":"2024-02-10T06:44:44.984801+0000","last_peered":"2024-02-10T06:44:44.984801+0000","last_clean":"2024-02-10T06:44:44.984801+0000","last_became_active":"2024-02-10T06:44:05.197163+0000","last_became_peered":"2024-02-10T06:44:05.197163+0000","last_unstale":"2024-02-10T06:44:44.984801+0000","last_undegraded":"2024-02-10T06:44:44.984801+0000","last_fullsized":"2024-02-10T06:44:44.984801+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:35:28.665783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919088+0000","last_change":"2024-02-10T06:44:07.198598+0000","last_active":"2024-02-10T06:44:44.919088+0000","last_peered":"2024-02-10T06:44:44.919088+0000","last_clean":"2024-02-10T06:44:44.919088+0000","last_became_active":"2024-02-10T06:44:07.198304+0000","last_became_peered":"2024-02-10T06:44:07.198304+0000","last_unstale":"2024-02-10T06:44:44.919088+0000","last_undegraded":"2024-02-10T06:44:44.919088+0000","last_fullsized":"2024-02-10T06:44:44.919088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:19:30.304295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.871359+0000","last_change":"2024-02-10T06:44:09.215393+0000","last_active":"2024-02-10T06:44:44.871359+0000","last_peered":"2024-02-10T06:44:44.871359+0000","last_clean":"2024-02-10T06:44:44.871359+0000","last_became_active":"2024-02-10T06:44:09.215118+0000","last_became_peered":"2024-02-10T06:44:09.215118+0000","last_unstale":"2024-02-10T06:44:44.871359+0000","last_undegraded":"2024-02-10T06:44:44.871359+0000","last_fullsized":"2024-02-10T06:44:44.871359+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:30:47.762572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"64'2","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868155+0000","last_change":"2024-02-10T06:44:07.187187+0000","last_active":"2024-02-10T06:44:44.868155+0000","last_peered":"2024-02-10T06:44:44.868155+0000","last_clean":"2024-02-10T06:44:44.868155+0000","last_became_active":"2024-02-10T06:44:03.136155+0000","last_became_peered":"2024-02-10T06:44:03.136155+0000","last_unstale":"2024-02-10T06:44:44.868155+0000","last_undegraded":"2024-02-10T06:44:44.868155+0000","last_fullsized":"2024-02-10T06:44:44.868155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:31:40.233102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055916500000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870241+0000","last_change":"2024-02-10T06:44:03.130646+0000","last_active":"2024-02-10T06:44:44.870241+0000","last_peered":"2024-02-10T06:44:44.870241+0000","last_clean":"2024-02-10T06:44:44.870241+0000","last_became_active":"2024-02-10T06:44:03.130367+0000","last_became_peered":"2024-02-10T06:44:03.130367+0000","last_unstale":"2024-02-10T06:44:44.870241+0000","last_undegraded":"2024-02-10T06:44:44.870241+0000","last_fullsized":"2024-02-10T06:44:44.870241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:02:53.864472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":276,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:46.968932+0000","last_change":"2024-02-10T06:43:53.469413+0000","last_active":"2024-02-10T06:44:46.968932+0000","last_peered":"2024-02-10T06:44:46.968932+0000","last_clean":"2024-02-10T06:44:46.968932+0000","last_became_active":"2024-02-10T06:43:53.468956+0000","last_became_peered":"2024-02-10T06:43:53.468956+0000","last_unstale":"2024-02-10T06:44:46.968932+0000","last_undegraded":"2024-02-10T06:44:46.968932+0000","last_fullsized":"2024-02-10T06:44:46.968932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:41:41.393633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:41:41.393633+0000","last_clean_scrub_stamp":"2024-02-10T06:41:41.393633+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:36:26.201690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"67'16","reported_seq":69,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194730+0000","last_change":"2024-02-10T06:44:05.332613+0000","last_active":"2024-02-10T06:45:23.194730+0000","last_peered":"2024-02-10T06:45:23.194730+0000","last_clean":"2024-02-10T06:45:23.194730+0000","last_became_active":"2024-02-10T06:44:05.332311+0000","last_became_peered":"2024-02-10T06:44:05.332311+0000","last_unstale":"2024-02-10T06:45:23.194730+0000","last_undegraded":"2024-02-10T06:45:23.194730+0000","last_fullsized":"2024-02-10T06:45:23.194730+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:06:42.524661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495420+0000","last_change":"2024-02-10T06:44:07.209631+0000","last_active":"2024-02-10T06:44:45.495420+0000","last_peered":"2024-02-10T06:44:45.495420+0000","last_clean":"2024-02-10T06:44:45.495420+0000","last_became_active":"2024-02-10T06:44:07.209262+0000","last_became_peered":"2024-02-10T06:44:07.209262+0000","last_unstale":"2024-02-10T06:44:45.495420+0000","last_undegraded":"2024-02-10T06:44:45.495420+0000","last_fullsized":"2024-02-10T06:44:45.495420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:31:32.377481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983821+0000","last_change":"2024-02-10T06:44:09.208268+0000","last_active":"2024-02-10T06:44:44.983821+0000","last_peered":"2024-02-10T06:44:44.983821+0000","last_clean":"2024-02-10T06:44:44.983821+0000","last_became_active":"2024-02-10T06:44:09.207977+0000","last_became_peered":"2024-02-10T06:44:09.207977+0000","last_unstale":"2024-02-10T06:44:44.983821+0000","last_undegraded":"2024-02-10T06:44:44.983821+0000","last_fullsized":"2024-02-10T06:44:44.983821+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:33:34.844848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867199+0000","last_change":"2024-02-10T06:44:07.193931+0000","last_active":"2024-02-10T06:44:44.867199+0000","last_peered":"2024-02-10T06:44:44.867199+0000","last_clean":"2024-02-10T06:44:44.867199+0000","last_became_active":"2024-02-10T06:44:03.137349+0000","last_became_peered":"2024-02-10T06:44:03.137349+0000","last_unstale":"2024-02-10T06:44:44.867199+0000","last_undegraded":"2024-02-10T06:44:44.867199+0000","last_fullsized":"2024-02-10T06:44:44.867199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:00:24.379348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002247083,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494256+0000","last_change":"2024-02-10T06:44:03.139424+0000","last_active":"2024-02-10T06:44:45.494256+0000","last_peered":"2024-02-10T06:44:45.494256+0000","last_clean":"2024-02-10T06:44:45.494256+0000","last_became_active":"2024-02-10T06:44:03.139090+0000","last_became_peered":"2024-02-10T06:44:03.139090+0000","last_unstale":"2024-02-10T06:44:45.494256+0000","last_undegraded":"2024-02-10T06:44:45.494256+0000","last_fullsized":"2024-02-10T06:44:45.494256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:50:52.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868241+0000","last_change":"2024-02-10T06:44:05.195965+0000","last_active":"2024-02-10T06:44:44.868241+0000","last_peered":"2024-02-10T06:44:44.868241+0000","last_clean":"2024-02-10T06:44:44.868241+0000","last_became_active":"2024-02-10T06:44:05.195644+0000","last_became_peered":"2024-02-10T06:44:05.195644+0000","last_unstale":"2024-02-10T06:44:44.868241+0000","last_undegraded":"2024-02-10T06:44:44.868241+0000","last_fullsized":"2024-02-10T06:44:44.868241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:52:00.789709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.871274+0000","last_change":"2024-02-10T06:44:07.202091+0000","last_active":"2024-02-10T06:44:44.871274+0000","last_peered":"2024-02-10T06:44:44.871274+0000","last_clean":"2024-02-10T06:44:44.871274+0000","last_became_active":"2024-02-10T06:44:07.201779+0000","last_became_peered":"2024-02-10T06:44:07.201779+0000","last_unstale":"2024-02-10T06:44:44.871274+0000","last_undegraded":"2024-02-10T06:44:44.871274+0000","last_fullsized":"2024-02-10T06:44:44.871274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:28:16.486258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983110+0000","last_change":"2024-02-10T06:44:09.209280+0000","last_active":"2024-02-10T06:44:44.983110+0000","last_peered":"2024-02-10T06:44:44.983110+0000","last_clean":"2024-02-10T06:44:44.983110+0000","last_became_active":"2024-02-10T06:44:09.208958+0000","last_became_peered":"2024-02-10T06:44:09.208958+0000","last_unstale":"2024-02-10T06:44:44.983110+0000","last_undegraded":"2024-02-10T06:44:44.983110+0000","last_fullsized":"2024-02-10T06:44:44.983110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:07:52.245853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495899+0000","last_change":"2024-02-10T06:44:03.133479+0000","last_active":"2024-02-10T06:44:45.495899+0000","last_peered":"2024-02-10T06:44:45.495899+0000","last_clean":"2024-02-10T06:44:45.495899+0000","last_became_active":"2024-02-10T06:44:03.133226+0000","last_became_peered":"2024-02-10T06:44:03.133226+0000","last_unstale":"2024-02-10T06:44:45.495899+0000","last_undegraded":"2024-02-10T06:44:45.495899+0000","last_fullsized":"2024-02-10T06:44:45.495899+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:01:35.101773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867287+0000","last_change":"2024-02-10T06:44:05.332232+0000","last_active":"2024-02-10T06:44:44.867287+0000","last_peered":"2024-02-10T06:44:44.867287+0000","last_clean":"2024-02-10T06:44:44.867287+0000","last_became_active":"2024-02-10T06:44:05.331961+0000","last_became_peered":"2024-02-10T06:44:05.331961+0000","last_unstale":"2024-02-10T06:44:44.867287+0000","last_undegraded":"2024-02-10T06:44:44.867287+0000","last_fullsized":"2024-02-10T06:44:44.867287+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:22:19.929930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496041+0000","last_change":"2024-02-10T06:44:07.199135+0000","last_active":"2024-02-10T06:44:45.496041+0000","last_peered":"2024-02-10T06:44:45.496041+0000","last_clean":"2024-02-10T06:44:45.496041+0000","last_became_active":"2024-02-10T06:44:07.198757+0000","last_became_peered":"2024-02-10T06:44:07.198757+0000","last_unstale":"2024-02-10T06:44:45.496041+0000","last_undegraded":"2024-02-10T06:44:45.496041+0000","last_fullsized":"2024-02-10T06:44:45.496041+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:21:24.785285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494551+0000","last_change":"2024-02-10T06:44:09.207729+0000","last_active":"2024-02-10T06:44:45.494551+0000","last_peered":"2024-02-10T06:44:45.494551+0000","last_clean":"2024-02-10T06:44:45.494551+0000","last_became_active":"2024-02-10T06:44:09.207432+0000","last_became_peered":"2024-02-10T06:44:09.207432+0000","last_unstale":"2024-02-10T06:44:45.494551+0000","last_undegraded":"2024-02-10T06:44:45.494551+0000","last_fullsized":"2024-02-10T06:44:45.494551+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T06:58:32.708616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"60'1","reported_seq":34,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984432+0000","last_change":"2024-02-10T06:44:03.137221+0000","last_active":"2024-02-10T06:44:44.984432+0000","last_peered":"2024-02-10T06:44:44.984432+0000","last_clean":"2024-02-10T06:44:44.984432+0000","last_became_active":"2024-02-10T06:44:03.136716+0000","last_became_peered":"2024-02-10T06:44:03.136716+0000","last_unstale":"2024-02-10T06:44:44.984432+0000","last_undegraded":"2024-02-10T06:44:44.984432+0000","last_fullsized":"2024-02-10T06:44:44.984432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:38:20.089065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867839+0000","last_change":"2024-02-10T06:44:05.332423+0000","last_active":"2024-02-10T06:44:44.867839+0000","last_peered":"2024-02-10T06:44:44.867839+0000","last_clean":"2024-02-10T06:44:44.867839+0000","last_became_active":"2024-02-10T06:44:05.332171+0000","last_became_peered":"2024-02-10T06:44:05.332171+0000","last_unstale":"2024-02-10T06:44:44.867839+0000","last_undegraded":"2024-02-10T06:44:44.867839+0000","last_fullsized":"2024-02-10T06:44:44.867839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:52:42.865648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494106+0000","last_change":"2024-02-10T06:44:07.201321+0000","last_active":"2024-02-10T06:44:45.494106+0000","last_peered":"2024-02-10T06:44:45.494106+0000","last_clean":"2024-02-10T06:44:45.494106+0000","last_became_active":"2024-02-10T06:44:07.200961+0000","last_became_peered":"2024-02-10T06:44:07.200961+0000","last_unstale":"2024-02-10T06:44:45.494106+0000","last_undegraded":"2024-02-10T06:44:45.494106+0000","last_fullsized":"2024-02-10T06:44:45.494106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:47:08.986191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"67'1","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495463+0000","last_change":"2024-02-10T06:44:09.213677+0000","last_active":"2024-02-10T06:44:45.495463+0000","last_peered":"2024-02-10T06:44:45.495463+0000","last_clean":"2024-02-10T06:44:45.495463+0000","last_became_active":"2024-02-10T06:44:09.213405+0000","last_became_peered":"2024-02-10T06:44:09.213405+0000","last_unstale":"2024-02-10T06:44:45.495463+0000","last_undegraded":"2024-02-10T06:44:45.495463+0000","last_fullsized":"2024-02-10T06:44:45.495463+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:16:35.414457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.496478+0000","last_change":"2024-02-10T06:44:03.133961+0000","last_active":"2024-02-10T06:44:45.496478+0000","last_peered":"2024-02-10T06:44:45.496478+0000","last_clean":"2024-02-10T06:44:45.496478+0000","last_became_active":"2024-02-10T06:44:03.133579+0000","last_became_peered":"2024-02-10T06:44:03.133579+0000","last_unstale":"2024-02-10T06:44:45.496478+0000","last_undegraded":"2024-02-10T06:44:45.496478+0000","last_fullsized":"2024-02-10T06:44:45.496478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:16:55.679227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":50,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493839+0000","last_change":"2024-02-10T06:44:05.192023+0000","last_active":"2024-02-10T06:44:45.493839+0000","last_peered":"2024-02-10T06:44:45.493839+0000","last_clean":"2024-02-10T06:44:45.493839+0000","last_became_active":"2024-02-10T06:44:05.191763+0000","last_became_peered":"2024-02-10T06:44:05.191763+0000","last_unstale":"2024-02-10T06:44:45.493839+0000","last_undegraded":"2024-02-10T06:44:45.493839+0000","last_fullsized":"2024-02-10T06:44:45.493839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:51:44.991293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194536+0000","last_change":"2024-02-10T06:44:07.202437+0000","last_active":"2024-02-10T06:45:23.194536+0000","last_peered":"2024-02-10T06:45:23.194536+0000","last_clean":"2024-02-10T06:45:23.194536+0000","last_became_active":"2024-02-10T06:44:07.202107+0000","last_became_peered":"2024-02-10T06:44:07.202107+0000","last_unstale":"2024-02-10T06:45:23.194536+0000","last_undegraded":"2024-02-10T06:45:23.194536+0000","last_fullsized":"2024-02-10T06:45:23.194536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:35:59.282975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.868216+0000","last_change":"2024-02-10T06:44:09.208220+0000","last_active":"2024-02-10T06:44:44.868216+0000","last_peered":"2024-02-10T06:44:44.868216+0000","last_clean":"2024-02-10T06:44:44.868216+0000","last_became_active":"2024-02-10T06:44:09.207939+0000","last_became_peered":"2024-02-10T06:44:09.207939+0000","last_unstale":"2024-02-10T06:44:44.868216+0000","last_undegraded":"2024-02-10T06:44:44.868216+0000","last_fullsized":"2024-02-10T06:44:44.868216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:07:16.019437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.871191+0000","last_change":"2024-02-10T06:44:07.200370+0000","last_active":"2024-02-10T06:44:44.871191+0000","last_peered":"2024-02-10T06:44:44.871191+0000","last_clean":"2024-02-10T06:44:44.871191+0000","last_became_active":"2024-02-10T06:44:07.200049+0000","last_became_peered":"2024-02-10T06:44:07.200049+0000","last_unstale":"2024-02-10T06:44:44.871191+0000","last_undegraded":"2024-02-10T06:44:44.871191+0000","last_fullsized":"2024-02-10T06:44:44.871191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:09:42.898197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494447+0000","last_change":"2024-02-10T06:44:03.136260+0000","last_active":"2024-02-10T06:44:45.494447+0000","last_peered":"2024-02-10T06:44:45.494447+0000","last_clean":"2024-02-10T06:44:45.494447+0000","last_became_active":"2024-02-10T06:44:03.135873+0000","last_became_peered":"2024-02-10T06:44:03.135873+0000","last_unstale":"2024-02-10T06:44:45.494447+0000","last_undegraded":"2024-02-10T06:44:45.494447+0000","last_fullsized":"2024-02-10T06:44:45.494447+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:16.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493503+0000","last_change":"2024-02-10T06:44:05.198080+0000","last_active":"2024-02-10T06:44:45.493503+0000","last_peered":"2024-02-10T06:44:45.493503+0000","last_clean":"2024-02-10T06:44:45.493503+0000","last_became_active":"2024-02-10T06:44:05.197796+0000","last_became_peered":"2024-02-10T06:44:05.197796+0000","last_unstale":"2024-02-10T06:44:45.493503+0000","last_undegraded":"2024-02-10T06:44:45.493503+0000","last_fullsized":"2024-02-10T06:44:45.493503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:32:47.398389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.870529+0000","last_change":"2024-02-10T06:44:09.213948+0000","last_active":"2024-02-10T06:44:44.870529+0000","last_peered":"2024-02-10T06:44:44.870529+0000","last_clean":"2024-02-10T06:44:44.870529+0000","last_became_active":"2024-02-10T06:44:09.213547+0000","last_became_peered":"2024-02-10T06:44:09.213547+0000","last_unstale":"2024-02-10T06:44:44.870529+0000","last_undegraded":"2024-02-10T06:44:44.870529+0000","last_fullsized":"2024-02-10T06:44:44.870529+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:15:53.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194726+0000","last_change":"2024-02-10T06:44:07.198041+0000","last_active":"2024-02-10T06:45:23.194726+0000","last_peered":"2024-02-10T06:45:23.194726+0000","last_clean":"2024-02-10T06:45:23.194726+0000","last_became_active":"2024-02-10T06:44:07.197737+0000","last_became_peered":"2024-02-10T06:44:07.197737+0000","last_unstale":"2024-02-10T06:45:23.194726+0000","last_undegraded":"2024-02-10T06:45:23.194726+0000","last_fullsized":"2024-02-10T06:45:23.194726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:49:22.956013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983303+0000","last_change":"2024-02-10T06:44:03.139934+0000","last_active":"2024-02-10T06:44:44.983303+0000","last_peered":"2024-02-10T06:44:44.983303+0000","last_clean":"2024-02-10T06:44:44.983303+0000","last_became_active":"2024-02-10T06:44:03.134239+0000","last_became_peered":"2024-02-10T06:44:03.134239+0000","last_unstale":"2024-02-10T06:44:44.983303+0000","last_undegraded":"2024-02-10T06:44:44.983303+0000","last_fullsized":"2024-02-10T06:44:44.983303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:21:03.100156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919208+0000","last_change":"2024-02-10T06:44:23.612907+0000","last_active":"2024-02-10T06:44:44.919208+0000","last_peered":"2024-02-10T06:44:44.919208+0000","last_clean":"2024-02-10T06:44:44.919208+0000","last_became_active":"2024-02-10T06:44:23.601024+0000","last_became_peered":"2024-02-10T06:44:23.601024+0000","last_unstale":"2024-02-10T06:44:44.919208+0000","last_undegraded":"2024-02-10T06:44:44.919208+0000","last_fullsized":"2024-02-10T06:44:44.919208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:03:23.937127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495378+0000","last_change":"2024-02-10T06:44:09.211740+0000","last_active":"2024-02-10T06:44:45.495378+0000","last_peered":"2024-02-10T06:44:45.495378+0000","last_clean":"2024-02-10T06:44:45.495378+0000","last_became_active":"2024-02-10T06:44:09.211444+0000","last_became_peered":"2024-02-10T06:44:09.211444+0000","last_unstale":"2024-02-10T06:44:45.495378+0000","last_undegraded":"2024-02-10T06:44:45.495378+0000","last_fullsized":"2024-02-10T06:44:45.495378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:05:18.228024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"67'8","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194496+0000","last_change":"2024-02-10T06:44:07.197665+0000","last_active":"2024-02-10T06:45:23.194496+0000","last_peered":"2024-02-10T06:45:23.194496+0000","last_clean":"2024-02-10T06:45:23.194496+0000","last_became_active":"2024-02-10T06:44:07.197348+0000","last_became_peered":"2024-02-10T06:44:07.197348+0000","last_unstale":"2024-02-10T06:45:23.194496+0000","last_undegraded":"2024-02-10T06:45:23.194496+0000","last_fullsized":"2024-02-10T06:45:23.194496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:48:34.244194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983351+0000","last_change":"2024-02-10T06:44:03.134744+0000","last_active":"2024-02-10T06:44:44.983351+0000","last_peered":"2024-02-10T06:44:44.983351+0000","last_clean":"2024-02-10T06:44:44.983351+0000","last_became_active":"2024-02-10T06:44:03.134431+0000","last_became_peered":"2024-02-10T06:44:03.134431+0000","last_unstale":"2024-02-10T06:44:44.983351+0000","last_undegraded":"2024-02-10T06:44:44.983351+0000","last_fullsized":"2024-02-10T06:44:44.983351+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:07:24.103517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493659+0000","last_change":"2024-02-10T06:44:05.192829+0000","last_active":"2024-02-10T06:44:45.493659+0000","last_peered":"2024-02-10T06:44:45.493659+0000","last_clean":"2024-02-10T06:44:45.493659+0000","last_became_active":"2024-02-10T06:44:05.192543+0000","last_became_peered":"2024-02-10T06:44:05.192543+0000","last_unstale":"2024-02-10T06:44:45.493659+0000","last_undegraded":"2024-02-10T06:44:45.493659+0000","last_fullsized":"2024-02-10T06:44:45.493659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:21:26.897748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919264+0000","last_change":"2024-02-10T06:44:09.209054+0000","last_active":"2024-02-10T06:44:44.919264+0000","last_peered":"2024-02-10T06:44:44.919264+0000","last_clean":"2024-02-10T06:44:44.919264+0000","last_became_active":"2024-02-10T06:44:09.208534+0000","last_became_peered":"2024-02-10T06:44:09.208534+0000","last_unstale":"2024-02-10T06:44:44.919264+0000","last_undegraded":"2024-02-10T06:44:44.919264+0000","last_fullsized":"2024-02-10T06:44:44.919264+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:17:47.646864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.983909+0000","last_change":"2024-02-10T06:44:07.209886+0000","last_active":"2024-02-10T06:44:44.983909+0000","last_peered":"2024-02-10T06:44:44.983909+0000","last_clean":"2024-02-10T06:44:44.983909+0000","last_became_active":"2024-02-10T06:44:07.209393+0000","last_became_peered":"2024-02-10T06:44:07.209393+0000","last_unstale":"2024-02-10T06:44:44.983909+0000","last_undegraded":"2024-02-10T06:44:44.983909+0000","last_fullsized":"2024-02-10T06:44:44.983909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:33:08.994086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495131+0000","last_change":"2024-02-10T06:44:03.134895+0000","last_active":"2024-02-10T06:44:45.495131+0000","last_peered":"2024-02-10T06:44:45.495131+0000","last_clean":"2024-02-10T06:44:45.495131+0000","last_became_active":"2024-02-10T06:44:03.134398+0000","last_became_peered":"2024-02-10T06:44:03.134398+0000","last_unstale":"2024-02-10T06:44:45.495131+0000","last_undegraded":"2024-02-10T06:44:45.495131+0000","last_fullsized":"2024-02-10T06:44:45.495131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:16:29.819393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.493476+0000","last_change":"2024-02-10T06:44:05.198764+0000","last_active":"2024-02-10T06:44:45.493476+0000","last_peered":"2024-02-10T06:44:45.493476+0000","last_clean":"2024-02-10T06:44:45.493476+0000","last_became_active":"2024-02-10T06:44:05.198450+0000","last_became_peered":"2024-02-10T06:44:05.198450+0000","last_unstale":"2024-02-10T06:44:45.493476+0000","last_undegraded":"2024-02-10T06:44:45.493476+0000","last_fullsized":"2024-02-10T06:44:45.493476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:12:12.836875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.494163+0000","last_change":"2024-02-10T06:44:09.213430+0000","last_active":"2024-02-10T06:44:45.494163+0000","last_peered":"2024-02-10T06:44:45.494163+0000","last_clean":"2024-02-10T06:44:45.494163+0000","last_became_active":"2024-02-10T06:44:09.212659+0000","last_became_peered":"2024-02-10T06:44:09.212659+0000","last_unstale":"2024-02-10T06:44:45.494163+0000","last_undegraded":"2024-02-10T06:44:45.494163+0000","last_fullsized":"2024-02-10T06:44:45.494163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:48:39.295872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918995+0000","last_change":"2024-02-10T06:44:07.195008+0000","last_active":"2024-02-10T06:44:44.918995+0000","last_peered":"2024-02-10T06:44:44.918995+0000","last_clean":"2024-02-10T06:44:44.918995+0000","last_became_active":"2024-02-10T06:44:07.194659+0000","last_became_peered":"2024-02-10T06:44:07.194659+0000","last_unstale":"2024-02-10T06:44:44.918995+0000","last_undegraded":"2024-02-10T06:44:44.918995+0000","last_fullsized":"2024-02-10T06:44:44.918995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:45:45.411029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919270+0000","last_change":"2024-02-10T06:44:03.139848+0000","last_active":"2024-02-10T06:44:44.919270+0000","last_peered":"2024-02-10T06:44:44.919270+0000","last_clean":"2024-02-10T06:44:44.919270+0000","last_became_active":"2024-02-10T06:44:03.139568+0000","last_became_peered":"2024-02-10T06:44:03.139568+0000","last_unstale":"2024-02-10T06:44:44.919270+0000","last_undegraded":"2024-02-10T06:44:44.919270+0000","last_fullsized":"2024-02-10T06:44:44.919270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:29:09.134748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:45.495768+0000","last_change":"2024-02-10T06:44:05.331722+0000","last_active":"2024-02-10T06:44:45.495768+0000","last_peered":"2024-02-10T06:44:45.495768+0000","last_clean":"2024-02-10T06:44:45.495768+0000","last_became_active":"2024-02-10T06:44:05.331226+0000","last_became_peered":"2024-02-10T06:44:05.331226+0000","last_unstale":"2024-02-10T06:44:45.495768+0000","last_undegraded":"2024-02-10T06:44:45.495768+0000","last_fullsized":"2024-02-10T06:44:45.495768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:29:42.502340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.866894+0000","last_change":"2024-02-10T06:44:09.206911+0000","last_active":"2024-02-10T06:44:44.866894+0000","last_peered":"2024-02-10T06:44:44.866894+0000","last_clean":"2024-02-10T06:44:44.866894+0000","last_became_active":"2024-02-10T06:44:09.206602+0000","last_became_peered":"2024-02-10T06:44:09.206602+0000","last_unstale":"2024-02-10T06:44:44.866894+0000","last_undegraded":"2024-02-10T06:44:44.866894+0000","last_fullsized":"2024-02-10T06:44:44.866894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:54:55.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.918948+0000","last_change":"2024-02-10T06:44:07.195594+0000","last_active":"2024-02-10T06:44:44.918948+0000","last_peered":"2024-02-10T06:44:44.918948+0000","last_clean":"2024-02-10T06:44:44.918948+0000","last_became_active":"2024-02-10T06:44:07.195286+0000","last_became_peered":"2024-02-10T06:44:07.195286+0000","last_unstale":"2024-02-10T06:44:44.918948+0000","last_undegraded":"2024-02-10T06:44:44.918948+0000","last_fullsized":"2024-02-10T06:44:44.918948+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:42:52.871553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"60'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.984514+0000","last_change":"2024-02-10T06:44:03.136088+0000","last_active":"2024-02-10T06:44:44.984514+0000","last_peered":"2024-02-10T06:44:44.984514+0000","last_clean":"2024-02-10T06:44:44.984514+0000","last_became_active":"2024-02-10T06:44:03.134812+0000","last_became_peered":"2024-02-10T06:44:03.134812+0000","last_unstale":"2024-02-10T06:44:44.984514+0000","last_undegraded":"2024-02-10T06:44:44.984514+0000","last_fullsized":"2024-02-10T06:44:44.984514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:37.446247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.867432+0000","last_change":"2024-02-10T06:44:05.196308+0000","last_active":"2024-02-10T06:44:44.867432+0000","last_peered":"2024-02-10T06:44:44.867432+0000","last_clean":"2024-02-10T06:44:44.867432+0000","last_became_active":"2024-02-10T06:44:05.196019+0000","last_became_peered":"2024-02-10T06:44:05.196019+0000","last_unstale":"2024-02-10T06:44:44.867432+0000","last_undegraded":"2024-02-10T06:44:44.867432+0000","last_fullsized":"2024-02-10T06:44:44.867432+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:48:58.667363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869890+0000","last_change":"2024-02-10T06:44:09.209515+0000","last_active":"2024-02-10T06:44:44.869890+0000","last_peered":"2024-02-10T06:44:44.869890+0000","last_clean":"2024-02-10T06:44:44.869890+0000","last_became_active":"2024-02-10T06:44:09.209202+0000","last_became_peered":"2024-02-10T06:44:09.209202+0000","last_unstale":"2024-02-10T06:44:44.869890+0000","last_undegraded":"2024-02-10T06:44:44.869890+0000","last_fullsized":"2024-02-10T06:44:44.869890+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:08.183633+0000","last_clean_scrub_stamp":"2024-02-10T06:44:08.183633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:58:28.030431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":33,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.919277+0000","last_change":"2024-02-10T06:44:03.137324+0000","last_active":"2024-02-10T06:44:44.919277+0000","last_peered":"2024-02-10T06:44:44.919277+0000","last_clean":"2024-02-10T06:44:44.919277+0000","last_became_active":"2024-02-10T06:44:03.136986+0000","last_became_peered":"2024-02-10T06:44:03.136986+0000","last_unstale":"2024-02-10T06:44:44.919277+0000","last_undegraded":"2024-02-10T06:44:44.919277+0000","last_fullsized":"2024-02-10T06:44:44.919277+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:02.106106+0000","last_clean_scrub_stamp":"2024-02-10T06:44:02.106106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:53:06.162086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:44:44.869856+0000","last_change":"2024-02-10T06:44:05.333020+0000","last_active":"2024-02-10T06:44:44.869856+0000","last_peered":"2024-02-10T06:44:44.869856+0000","last_clean":"2024-02-10T06:44:44.869856+0000","last_became_active":"2024-02-10T06:44:05.332656+0000","last_became_peered":"2024-02-10T06:44:05.332656+0000","last_unstale":"2024-02-10T06:44:44.869856+0000","last_undegraded":"2024-02-10T06:44:44.869856+0000","last_fullsized":"2024-02-10T06:44:44.869856+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:04.165695+0000","last_clean_scrub_stamp":"2024-02-10T06:44:04.165695+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:50:15.802954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"67'8","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-02-10T06:45:23.194731+0000","last_change":"2024-02-10T06:44:07.203447+0000","last_active":"2024-02-10T06:45:23.194731+0000","last_peered":"2024-02-10T06:45:23.194731+0000","last_clean":"2024-02-10T06:45:23.194731+0000","last_became_active":"2024-02-10T06:44:07.203138+0000","last_became_peered":"2024-02-10T06:44:07.203138+0000","last_unstale":"2024-02-10T06:45:23.194731+0000","last_undegraded":"2024-02-10T06:45:23.194731+0000","last_fullsized":"2024-02-10T06:45:23.194731+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T06:44:06.179994+0000","last_clean_scrub_stamp":"2024-02-10T06:44:06.179994+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:03:16.737044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":77,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234007,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32444,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712708,"statfs":{"total":95995035648,"available":95961812992,"internally_reserved":0,"allocated":1298432,"data_stored":867391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.23000000000000001}]},{"osd":1,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.27900000000000003}]},{"osd":2,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.49299999999999999}]},{"osd":6,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495644,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32440,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712712,"statfs":{"total":95995035648,"available":95961817088,"internally_reserved":0,"allocated":1294336,"data_stored":867210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.755,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71499999999999997,"5min":1.431,"15min":1.431},"last":0.58699999999999997}]},{"osd":1,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.69899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69799999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68300000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68300000000000005,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.504}]},{"osd":3,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67200000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72799999999999998,"5min":0.873,"15min":0.873},"last":0.54200000000000004}]},{"osd":4,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.28299999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64700000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.437,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42799999999999999}]},{"osd":5,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.44700000000000001,"15min":0.438},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":733184,"data_stored":278136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.40300000000000002,"15min":0.34599999999999997},"min":{"1min":0.27300000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.395,"5min":3.395,"15min":3.395},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.41499999999999998,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.2309999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.27100000000000002}]},{"osd":2,"last update":"Sat Feb 10 06:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.626,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.32000000000000001}]},{"osd":3,"last update":"Sat Feb 10 06:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.318,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.1539999999999999,"5min":3.1539999999999999,"15min":3.1539999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47899999999999998,"15min":0.45500000000000002},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.33400000000000002}]},{"osd":4,"last update":"Sat Feb 10 06:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.501},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48499999999999999}]},{"osd":5,"last update":"Sat Feb 10 06:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45300000000000001}]},{"osd":6,"last update":"Sat Feb 10 06:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Feb 10 06:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.432},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56499999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32448,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712704,"statfs":{"total":95995035648,"available":95961808896,"internally_reserved":0,"allocated":1302528,"data_stored":867995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 06:45:20 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36699999999999999,"15min":0.35499999999999998},"min":{"1min":0.22700000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.55200000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.433},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.29399999999999998}]},{"osd":2,"last update":"Sat Feb 10 06:44:48 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.41799999999999998,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61699999999999999,"5min":0.621,"15min":0.621},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.39600000000000002,"15min":0.38800000000000001},"min":{"1min":0.26600000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Sat Feb 10 06:45:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.42499999999999999,"15min":0.42599999999999999},"min":{"1min":0.29099999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59899999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.373}]},{"osd":4,"last update":"Sat Feb 10 06:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54400000000000004,"15min":0.53900000000000003},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.626},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75800000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Feb 10 06:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60899999999999999}]},{"osd":6,"last update":"Sat Feb 10 06:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68899999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.496},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.75,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat Feb 10 06:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717452,"statfs":{"total":95995035648,"available":95966670848,"internally_reserved":0,"allocated":696320,"data_stored":277084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.50700000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Sat Feb 10 06:44:44 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57599999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.72199999999999998}]},{"osd":3,"last update":"Sat Feb 10 06:45:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42499999999999999,"15min":0.42299999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53500000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45300000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65000000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.57599999999999996}]},{"osd":4,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.752,"15min":0.752},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":1.085}]},{"osd":5,"last update":"Sat Feb 10 06:45:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.873,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":1.073}]},{"osd":6,"last update":"Sat Feb 10 06:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":1.026}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":1.04},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":1.129}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":278530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:45:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.38600000000000001,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.41099999999999998,"15min":0.41699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.438}]},{"osd":1,"last update":"Sat Feb 10 06:45:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.41599999999999998,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54000000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Feb 10 06:45:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53900000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.377,"5min":0.433,"15min":0.441},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68899999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.76900000000000002}]},{"osd":4,"last update":"Sat Feb 10 06:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.70499999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.88300000000000001}]},{"osd":5,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":0.751,"15min":0.751},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Feb 10 06:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.748,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Feb 10 06:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.90800000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018918,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":278179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.70399999999999996}]},{"osd":1,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73099999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.76200000000000001}]},{"osd":2,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69499999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.75}]},{"osd":3,"last update":"Sat Feb 10 06:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Feb 10 06:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.33700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.746,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.41899999999999998,"15min":0.40999999999999998},"min":{"1min":0.33300000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.626}]},{"osd":6,"last update":"Sat Feb 10 06:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81899999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52300000000000002}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54700000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757282,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713296,"statfs":{"total":95995035648,"available":95962415104,"internally_reserved":0,"allocated":692224,"data_stored":276783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.61899999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.73699999999999999}]},{"osd":1,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.501,"15min":0.503},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.621,"5min":0.627,"15min":0.627},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.65200000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.66500000000000004}]},{"osd":2,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50800000000000001,"15min":0.51100000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53500000000000003,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.65900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.753},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Feb 10 06:45:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.33500000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat Feb 10 06:45:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.629},{"interface":"front","average":{"1min":0.5,"5min":0.47799999999999998,"15min":0.47399999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67100000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Feb 10 06:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T06:45:34.510 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-10T06:45:34.510 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-10T06:45:34.510 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-10T06:45:34.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid df845a14-c7de-11ee-95b8-87774f69a715 -- ceph health --format=json 2024-02-10T06:45:34.749 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T06:45:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:35 smithi174 ceph-mon[29878]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:35 smithi174 ceph-mon[39523]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:36.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:35 smithi184 ceph-mon[33513]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:36.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:45:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:45:36.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 -- 172.21.15.174:0/3463423250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0101d70 msgr2=0x7fcec01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:36.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 --2- 172.21.15.174:0/3463423250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0101d70 0x7fcec01021f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009990 tx=0x7fceb4009ca0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 -- 172.21.15.174:0/3463423250 shutdown_connections 2024-02-10T06:45:36.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 --2- 172.21.15.174:0/3463423250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0102730 0x7fcec010ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 --2- 172.21.15.174:0/3463423250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 0x7fcec0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 --2- 172.21.15.174:0/3463423250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0101d70 0x7fcec01021f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 -- 172.21.15.174:0/3463423250 >> 172.21.15.174:0/3463423250 conn(0x7fcec0078e50 msgr2=0x7fcec0079260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:36.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 -- 172.21.15.174:0/3463423250 shutdown_connections 2024-02-10T06:45:36.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.517+0000 7fcec6538700 1 -- 172.21.15.174:0/3463423250 wait complete. 2024-02-10T06:45:36.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.518+0000 7fcec6538700 1 Processor -- start 2024-02-10T06:45:36.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.518+0000 7fcec6538700 1 -- start start 2024-02-10T06:45:36.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec6538700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0101d70 0x7fcec019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:36.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec6538700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0102730 0x7fcec019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:36.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec6538700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 0x7fcec0197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec6538700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0197f30 con 0x7fcec0107e20 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec6538700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec01980a0 con 0x7fcec0101d70 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebf7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0102730 0x7fcec019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0101d70 0x7fcec019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebf7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0102730 0x7fcec019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:54062/0 (socket says 172.21.15.174:54062) 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebf7fe700 1 -- 172.21.15.174:0/2936256361 learned_addr learned my addr 172.21.15.174:0/2936256361 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0198260 con 0x7fcec0102730 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec4ad5700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 0x7fcec0197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebffff700 1 -- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0102730 msgr2=0x7fcec019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebffff700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0102730 0x7fcec019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebffff700 1 -- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 msgr2=0x7fcec0197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebffff700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 0x7fcec0197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcebffff700 1 -- 172.21.15.174:0/2936256361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb0009520 con 0x7fcec0101d70 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.519+0000 7fcec4ad5700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 0x7fcec0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.520+0000 7fcebffff700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0101d70 0x7fcec019d660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9f0 tx=0x7fceb000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:36.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.520+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb000cb70 con 0x7fcec0101d70 2024-02-10T06:45:36.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.520+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fceb000ee90 con 0x7fcec0101d70 2024-02-10T06:45:36.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.520+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb40095f0 con 0x7fcec0101d70 2024-02-10T06:45:36.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.520+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0003c40 con 0x7fcec0101d70 2024-02-10T06:45:36.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.520+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec01a45e0 con 0x7fcec0101d70 2024-02-10T06:45:36.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.521+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec0103460 con 0x7fcec0101d70 2024-02-10T06:45:36.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.521+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fceb0004020 con 0x7fcec0101d70 2024-02-10T06:45:36.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.522+0000 7fcebd7fa700 1 --2- 172.21.15.174:0/2936256361 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcea806da80 0x7fcea806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T06:45:36.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.522+0000 7fcebf7fe700 1 --2- 172.21.15.174:0/2936256361 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcea806da80 0x7fcea806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T06:45:36.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.522+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fceb0014070 con 0x7fcec0101d70 2024-02-10T06:45:36.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.522+0000 7fcebf7fe700 1 --2- 172.21.15.174:0/2936256361 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcea806da80 0x7fcea806ff40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcec0198ef0 tx=0x7fceb400b610 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T06:45:36.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.526+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fceb005ab10 con 0x7fcec0101d70 2024-02-10T06:45:36.769 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.768+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcec0198c60 con 0x7fcec0101d70 2024-02-10T06:45:36.769 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.768+0000 7fcebd7fa700 1 -- 172.21.15.174:0/2936256361 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fceb005a6a0 con 0x7fcec0101d70 2024-02-10T06:45:36.769 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-02-10T06:45:36.769 INFO:teuthology.orchestra.run.smithi174.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-10T06:45:36.771 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcea806da80 msgr2=0x7fcea806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:36.771 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 --2- 172.21.15.174:0/2936256361 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcea806da80 0x7fcea806ff40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcec0198ef0 tx=0x7fceb400b610 comp rx=0 tx=0).stop 2024-02-10T06:45:36.771 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0101d70 msgr2=0x7fcec019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T06:45:36.771 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0101d70 0x7fcec019d660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9f0 tx=0x7fceb000ed00 comp rx=0 tx=0).stop 2024-02-10T06:45:36.771 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 shutdown_connections 2024-02-10T06:45:36.771 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec0107e20 0x7fcec0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 --2- 172.21.15.174:0/2936256361 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcea806da80 0x7fcea806ff40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec0101d70 0x7fcec019d660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 --2- 172.21.15.174:0/2936256361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcec0102730 0x7fcec019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T06:45:36.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.770+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 >> 172.21.15.174:0/2936256361 conn(0x7fcec0078e50 msgr2=0x7fcec0105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T06:45:36.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.771+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 shutdown_connections 2024-02-10T06:45:36.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-02-10T06:45:36.771+0000 7fcec6538700 1 -- 172.21.15.174:0/2936256361 wait complete. 2024-02-10T06:45:37.138 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:36 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:37.138 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2936256361' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T06:45:37.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:36 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:37.139 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2936256361' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T06:45:37.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:36 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2936256361' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T06:45:37.566 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-10T06:45:37.566 INFO:tasks.cephadm:Setup complete, yielding 2024-02-10T06:45:37.567 INFO:teuthology.run_tasks:Running task workunit... 2024-02-10T06:45:37.581 INFO:tasks.workunit:Pulling workunits from ref 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:45:37.582 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-10T06:45:37.582 DEBUG:teuthology.orchestra.run.smithi174:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-10T06:45:37.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T06:45:37.597 INFO:teuthology.orchestra.run.smithi174.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-10T06:45:37.598 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-10T06:45:37.653 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-10T06:45:37.653 DEBUG:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-10T06:45:37.708 INFO:tasks.workunit:timeout=3h 2024-02-10T06:45:37.708 INFO:tasks.workunit:cleanup=True 2024-02-10T06:45:37.708 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T06:45:37.763 INFO:tasks.workunit.client.0.smithi174.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-10T06:45:38.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:37 smithi174 ceph-mon[29878]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:37 smithi174 ceph-mon[39523]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:45:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:45:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:37 smithi184 ceph-mon[33513]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:39 smithi174 ceph-mon[29878]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:40.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:39 smithi174 ceph-mon[39523]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:40.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:39 smithi184 ceph-mon[33513]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:41.239 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:45:40 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:45:40.794Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001711149s 2024-02-10T06:45:41.988 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:41 smithi184 ceph-mon[33513]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:42.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:41 smithi174 ceph-mon[29878]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:42.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:41 smithi174 ceph-mon[39523]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:44.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:43 smithi184 ceph-mon[33513]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:44.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:43 smithi174 ceph-mon[29878]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:43 smithi174 ceph-mon[39523]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:44.577 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 podman[66280]: 2024-02-10 06:45:44.463505653 +0000 UTC m=+0.119088651 container create 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a, maintainer=Paul Cuzner , io.buildah.version=1.32.0, vcs-type=git, vendor=Red Hat, Inc., version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, name=ubi8, description=Ceph Grafana Container, com.redhat.component=ubi8-container, release=239.1651231664, distribution-scope=public, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-10T06:45:44.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 podman[66280]: 2024-02-10 06:45:44.682786058 +0000 UTC m=+0.338369060 container init 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a, distribution-scope=public, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, vendor=Red Hat, Inc., version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, release=239.1651231664, architecture=x86_64, io.openshift.tags=base rhel8, name=ubi8, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , io.buildah.version=1.32.0) 2024-02-10T06:45:44.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 podman[66280]: 2024-02-10 06:45:44.688390925 +0000 UTC m=+0.343973913 container start 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a, architecture=x86_64, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, io.openshift.expose-services=, vendor=Red Hat, Inc., io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, vcs-type=git) 2024-02-10T06:45:44.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 bash[66280]: 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 2024-02-10T06:45:44.935 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 systemd[1]: Started Ceph grafana.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841561104Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.84174439Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.84176401Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841771191Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.84177744Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841783462Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841789675Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.84179553Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841801365Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841807046Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841812431Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841817786Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-10T06:45:44.936 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841823836Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=settings t=2024-02-10T06:45:44.841829725Z level=info msg="App mode production" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=sqlstore t=2024-02-10T06:45:44.84187126Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=sqlstore t=2024-02-10T06:45:44.8418867Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.842705701Z level=info msg="Starting DB migrations" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.842803071Z level=info msg="Executing migration" id="create migration_log table" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.843325682Z level=info msg="Executing migration" id="create user table" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.843720155Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.846649178Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.849379805Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-10T06:45:44.937 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.851955765Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.854354146Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.856842088Z level=info msg="Executing migration" id="create user table v2" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.859169254Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.861902297Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.864543674Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.867226108Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.87014347Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.871021899Z level=info msg="Executing migration" id="Update user table charset" 2024-02-10T06:45:44.938 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.871723528Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.872504605Z level=info msg="Executing migration" id="Add missing user data" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.872894817Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.873370343Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.874335144Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.874893333Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.876029599Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.877123743Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.877741949Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.87868575Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.879442778Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.880294825Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.881260336Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.881940659Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.882906644Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-10T06:45:44.939 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.88364761Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.884488792Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.885187058Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.885641464Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.886432943Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.886885821Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.887307407Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.888333999Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.889028023Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.889385642Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.890251556Z level=info msg="Executing migration" id="create star table" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.890646492Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.891175928Z level=info msg="Executing migration" id="create org table v1" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.892008849Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-10T06:45:44.940 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.892969606Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.893870891Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.894580932Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.895016967Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.895876789Z level=info msg="Executing migration" id="Update org table charset" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.89876971Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.899242196Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.901543644Z level=info msg="Executing migration" id="create dashboard table" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.902145957Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.905092183Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.908032923Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.910709469Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.911175542Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.914079942Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-10T06:45:44.941 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.916977593Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.919809818Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.920285958Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.923233151Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.925919486Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.928822016Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.929145194Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.931982317Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-10T06:45:44.942 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.935019319Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.937594722Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.938051087Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.940706926Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.94331156Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.946040312Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.946389575Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.949142445Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.949739567Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.951929818Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.954480301Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-10T06:45:45.188 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.955054091Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.957833276Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.960224936Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.962818303Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.965544874Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.968141086Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.968533076Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.969431639Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.970285773Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.971134568Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.972031028Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.972720104Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.97351792Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-10T06:45:45.189 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.974093362Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.974580003Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.974855538Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.975133971Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.975553383Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.977753686Z level=info msg="Executing migration" id="create data_source table" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.978246756Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.980542451Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.980970329Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.98136944Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.984213392Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.986978408Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.989740028Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.990169128Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-10T06:45:45.190 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.993106798Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.99576971Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.998493934Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:44.999064416Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.00179975Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.00452241Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.007276976Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.007589968Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.010362174Z level=info msg="Executing migration" id="Add uid column" 2024-02-10T06:45:45.192 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.011132389Z level=info msg="Executing migration" id="Update uid value" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.013018873Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.015719896Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.016175093Z level=info msg="Executing migration" id="create api_key table" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.018818403Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.019688785Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.020627734Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.021201622Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.022006018Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.022963005Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.023723233Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.024794178Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.025597162Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.026552949Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.027191782Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.028012709Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-10T06:45:45.193 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.028821781Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.02920359Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.030475642Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.031096556Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.031924132Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.032886053Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.033604802Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.034414204Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.034834325Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.035270484Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.035673322Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.036150276Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.036995778Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.038089275Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.038720518Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-10T06:45:45.194 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.03951401Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.040138952Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.040745003Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.041545803Z level=info msg="Executing migration" id="create quota table v1" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.041961223Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.042440832Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.043237088Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.043623159Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.044635071Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.045353385Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.046175555Z level=info msg="Executing migration" id="create session table" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.046613396Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.046914237Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.049438189Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.049879261Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.052650393Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-10T06:45:45.195 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.052974873Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.053273594Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.053565789Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.053835366Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.055519457Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.055815694Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.058271779Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.061146221Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.061872477Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.064602243Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.067298053Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.067785643Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.070625846Z level=info msg="Executing migration" id="create alert table v1" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.07353176Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.076163887Z level=info msg="Executing migration" id="add index alert state" 2024-02-10T06:45:45.196 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.07886629Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.079330275Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.079707974Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.080123582Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.080538947Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.081929767Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.082329089Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.082759199Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.083070322Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.083450371Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.083802159Z level=info msg="Executing migration" id="Add column is_default" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.084456821Z level=info msg="Executing migration" id="Add column frequency" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.085110812Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-10T06:45:45.197 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.085772856Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.086439857Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.086870671Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.087133037Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.087431383Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.087826823Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.088267393Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.088647345Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.08901123Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.089456329Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.09009313Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.090763688Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-10T06:45:45.198 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.09108679Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.091523141Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.091898364Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.09256974Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.092871334Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.093323334Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.093742987Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.094024805Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.094462033Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.094907965Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.095361687Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.095792031Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.097331488Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.099984957Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.102753804Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-10T06:45:45.199 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.10346722Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.106417671Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.1092028Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.111975613Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.114893857Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.117647483Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.120365724Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.123181538Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.125837777Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.128513162Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.131270372Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.134129876Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.13678345Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-10T06:45:45.200 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.139428172Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.142245013Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.144906295Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.147624889Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.150258221Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.153108482Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.155788662Z level=info msg="Executing migration" id="Move region to single row" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.158528493Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.161129912Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.162084158Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.162682128Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.163511795Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.163930408Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-10T06:45:45.201 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.16437314Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.165228117Z level=info msg="Executing migration" id="create test_data table" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.166227364Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.166942079Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.167880926Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.168679335Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.169635545Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.170428459Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.171184028Z level=info msg="Executing migration" id="create team table" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.172029119Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.17293477Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.173874336Z level=info msg="Executing migration" id="create team member table" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.174644185Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.17547313Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.176402623Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.177184999Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-10T06:45:45.202 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.178043055Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-10T06:45:45.203 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.179040475Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-10T06:45:45.203 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.181831319Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-10T06:45:45.203 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.184810303Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-10T06:45:45.203 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.187460409Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-10T06:45:45.203 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.190112498Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.192869595Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.195741521Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.198510307Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.198956155Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.201749383Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.204598537Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.207399542Z level=info msg="Executing migration" id="create tag table" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.207839212Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.210551479Z level=info msg="Executing migration" id="create login attempt table" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.213323048Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.216278136Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.216796351Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.219551486Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-10T06:45:45.438 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.222617989Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-10T06:45:45.439 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.225385302Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-10T06:45:45.439 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.225826981Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-10T06:45:45.439 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.228560612Z level=info msg="Executing migration" id="create user auth table" 2024-02-10T06:45:45.439 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.228970344Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-10T06:45:45.439 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.231222292Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.233927311Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.234681962Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.237420183Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.240014799Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.242775348Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.243319923Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.246027403Z level=info msg="Executing migration" id="create server_lock table" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.248791323Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-10T06:45:45.440 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.251449064Z level=info msg="Executing migration" id="create user auth token table" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.251883467Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.254387581Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.257212694Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.258028541Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.2588189Z level=info msg="Executing migration" id="create cache_data table" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.259662858Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.260482568Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.261479965Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.262115262Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.262986119Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.263728Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.264713066Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.265309968Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-10T06:45:45.441 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.266175278Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.266992463Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.267982219Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.268558005Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.269350117Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.269758912Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.270585335Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.271497038Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.27178668Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.272698944Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.273360042Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.273760386Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.274664669Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.275105745Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-10T06:45:45.442 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.275502407Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.276320836Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.276732355Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.27754432Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.27842393Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.278843377Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.282291986Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.285615885Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.286456636Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.28688006Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.2877898Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.290652349Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.293387739Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.293824662Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-10T06:45:45.443 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.296554262Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.299441942Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.3022056Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.303012583Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.305809609Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.308615334Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.311319586Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.312120746Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.314976343Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.317547116Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.320329221Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.321307898Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-10T06:45:45.444 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.323911948Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.326780474Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.329532929Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.329969113Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.33262576Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.33531163Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.338042465Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.340771532Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.343432309Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.346216587Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.348792269Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.349195396Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.351818443Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.354415652Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.356918241Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.357368968Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-10T06:45:45.445 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.360200292Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.362854413Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.365284187Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.366299228Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.36715348Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.368116435Z level=info msg="Executing migration" id="create alert_image table" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.368823478Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.369252116Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.370201954Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.370974205Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.37194368Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.372074332Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T06:45:45.446 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.37234826Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.373192851Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.374031173Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.374152835Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.374260719Z level=info msg="alerts found to migrate" alerts=0 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.37479119Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.375223676Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.376237611Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.376925264Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.377912451Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.378471942Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.379404705Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.380132226Z level=info msg="Executing migration" id="create data_keys table" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.381110307Z level=info msg="Executing migration" id="create secrets table" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.381529358Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.385701303Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.386643009Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-10T06:45:45.447 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.387482498Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.391592442Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.395883631Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.396797967Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.397602491Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.397940812Z level=info msg="Executing migration" id="create permission table" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.398846084Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-10T06:45:45.448 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.399693807Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.400521549Z level=info msg="Executing migration" id="create role table" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.400916351Z level=info msg="Executing migration" id="add column display_name" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.401837775Z level=info msg="Executing migration" id="add column group_name" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.402767978Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.403767829Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.404364926Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.407365087Z level=info msg="Executing migration" id="create team role table" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.410200611Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.413012476Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.413489499Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.416450135Z level=info msg="Executing migration" id="create user role table" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.419308895Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.422025571Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.424704149Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.427510294Z level=info msg="Executing migration" id="create builtin role table" 2024-02-10T06:45:45.449 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.430306604Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.433072153Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.43351065Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.436220748Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.436756559Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.437251368Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.437713124Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.439393397Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-10T06:45:45.450 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.439892051Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.442192241Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.443380978Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.44383937Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.444343804Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.444666994Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.44511562Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.445648854Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.446066672Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.446387174Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.446707826Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-10T06:45:45.829 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.447094755Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.447524671Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.44848029Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.448775914Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.449174158Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.44959735Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.450039598Z level=info msg="Executing migration" id="add correlation config column" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.451014269Z level=info msg="Executing migration" id="create entity_events table" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.451464316Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.454231003Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.454386208Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.454678459Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.454754131Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.454992986Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-10T06:45:45.830 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.45542368Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.455835877Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.456265777Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.456689663Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.457083016Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.457498755Z level=info msg="Executing migration" id="Drop public config table" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.457897836Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.458339351Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.458780781Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.45923323Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.459659554Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-10T06:45:45.831 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.462629346Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.463616409Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.464598766Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.464909053Z level=info msg="Executing migration" id="add share column" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.465902749Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.466345023Z level=info msg="Executing migration" id="create file table" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.466778457Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-10T06:45:45.832 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.467219257Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.467688325Z level=info msg="Executing migration" id="create file_meta table" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.470504299Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.472953879Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.473274151Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.476141788Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.478904424Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.481538443Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.482624646Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.485331999Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.488220233Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.490823331Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.491164999Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.494022974Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-10T06:45:45.833 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.496726402Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.499239414Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.499674945Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.509192766Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.511860631Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.512835329Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.516249069Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.51709334Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.518015911Z level=info msg="Executing migration" id="create folder table" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.518767495Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.519208807Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.520146713Z level=info msg="Executing migration" id="Update folder title length" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.520821344Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=migrator t=2024-02-10T06:45:45.521733939Z level=info msg="migrations completed" performed=485 skipped=0 duration=678.973412ms 2024-02-10T06:45:45.834 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=sqlstore t=2024-02-10T06:45:45.522340639Z level=info msg="Created default organization" 2024-02-10T06:45:45.835 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=plugin.loader t=2024-02-10T06:45:45.577352688Z level=info msg="Plugin registered" pluginID=input 2024-02-10T06:45:45.835 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=plugin.loader t=2024-02-10T06:45:45.604319659Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-10T06:45:45.835 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=plugin.loader t=2024-02-10T06:45:45.604341143Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-10T06:45:45.835 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=secrets t=2024-02-10T06:45:45.604372395Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-10T06:45:45.835 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=query_data t=2024-02-10T06:45:45.606411978Z level=info msg="Query Service initialization" 2024-02-10T06:45:45.835 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=live.push_http t=2024-02-10T06:45:45.608765622Z level=info msg="Live Push Gateway initialization" 2024-02-10T06:45:46.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: Deploying daemon node-exporter.a on smithi174 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: Deploying daemon node-exporter.a on smithi174 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: Deploying daemon node-exporter.a on smithi174 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:45:46.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:46 smithi174 bash[77910]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T06:45:46.544 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=infra.usagestats.collector t=2024-02-10T06:45:46.21314251Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-10T06:45:46.544 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=provisioning.datasources t=2024-02-10T06:45:46.214177033Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-10T06:45:46.544 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=provisioning.datasources t=2024-02-10T06:45:46.224942705Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=provisioning.alerting t=2024-02-10T06:45:46.235555903Z level=info msg="starting to provision alerting" 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=provisioning.alerting t=2024-02-10T06:45:46.235579211Z level=info msg="finished to provision alerting" 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=grafanaStorageLogger t=2024-02-10T06:45:46.23616392Z level=info msg="storage starting" 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=ngalert.state.manager t=2024-02-10T06:45:46.240388141Z level=info msg="Warming state cache for startup" 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=ngalert.state.manager t=2024-02-10T06:45:46.240871618Z level=info msg="State cache has been initialized" states=0 duration=481.696µs 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=ticker t=2024-02-10T06:45:46.240999788Z level=info msg=starting first_tick=2024-02-10T06:45:50Z 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=ngalert.multiorg.alertmanager t=2024-02-10T06:45:46.241160515Z level=info msg="starting MultiOrg Alertmanager" 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=http.server t=2024-02-10T06:45:46.242258233Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-10T06:45:46.545 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=sqlstore.transactions t=2024-02-10T06:45:46.252043861Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-10T06:45:46.846 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:46.846 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:46.846 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:46 smithi174 bash[77910]: Getting image source signatures 2024-02-10T06:45:46.846 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:46 smithi174 bash[77910]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T06:45:46.847 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:46 smithi174 bash[77910]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T06:45:46.847 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:46 smithi174 bash[77910]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T06:45:47.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:48.088 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:47 smithi174 ceph-mon[29878]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:48.088 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:45:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:45:48.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:47 smithi174 ceph-mon[39523]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:48.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:47 smithi184 ceph-mon[33513]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:48.488 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 bash[77910]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T06:45:48.489 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 bash[77910]: Writing manifest to image destination 2024-02-10T06:45:48.903 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 podman[77910]: 2024-02-10 06:45:48.502839756 +0000 UTC m=+2.542785907 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T06:45:48.903 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 podman[77910]: 2024-02-10 06:45:48.618071119 +0000 UTC m=+2.658017318 container create 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:45:48.904 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 podman[77910]: 2024-02-10 06:45:48.903771126 +0000 UTC m=+2.943717270 container init 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:45:49.239 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 podman[77910]: 2024-02-10 06:45:48.909894206 +0000 UTC m=+2.949840338 container start 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T06:45:49.239 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.914Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T06:45:49.239 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.914Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T06:45:49.239 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T06:45:49.239 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T06:45:49.239 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T06:45:49.240 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T06:45:49.244 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.915Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T06:45:49.245 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a[77958]: ts=2024-02-10T06:45:48.916Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:49 smithi174 bash[77910]: 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 2024-02-10T06:45:49.246 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 06:45:49 smithi174 systemd[1]: Started Ceph node-exporter.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:45:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:50 smithi184 ceph-mon[33513]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:50 smithi184 ceph-mon[33513]: Deploying daemon node-exporter.b on smithi184 2024-02-10T06:45:50.373 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:50 smithi184 bash[66571]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[39523]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[39523]: Deploying daemon node-exporter.b on smithi184 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[29878]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:50 smithi174 ceph-mon[29878]: Deploying daemon node-exporter.b on smithi184 2024-02-10T06:45:50.872 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:50 smithi184 bash[66571]: Getting image source signatures 2024-02-10T06:45:50.873 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:50 smithi184 bash[66571]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T06:45:50.873 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:50 smithi184 bash[66571]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T06:45:50.873 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:50 smithi184 bash[66571]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T06:45:52.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-mon[33513]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:52.360 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 bash[66571]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T06:45:52.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:52 smithi174 ceph-mon[29878]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:52 smithi174 ceph-mon[39523]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:52.622 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 bash[66571]: Writing manifest to image destination 2024-02-10T06:45:52.941 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 podman[66571]: 2024-02-10 06:45:52.582763274 +0000 UTC m=+2.436985134 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T06:45:52.941 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 podman[66571]: 2024-02-10 06:45:52.689718277 +0000 UTC m=+2.543940137 container create d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:45:52.941 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 podman[66571]: 2024-02-10 06:45:52.941544374 +0000 UTC m=+2.795766234 container init d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:45:53.373 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 podman[66571]: 2024-02-10 06:45:52.947344036 +0000 UTC m=+2.801565899 container start d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T06:45:53.373 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.951Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T06:45:53.373 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.951Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T06:45:53.373 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.951Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T06:45:53.373 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T06:45:53.374 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T06:45:53.375 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T06:45:53.376 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.952Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b[66622]: ts=2024-02-10T06:45:52.953Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:53 smithi184 bash[66571]: d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 2024-02-10T06:45:53.377 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 06:45:53 smithi184 systemd[1]: Started Ceph node-exporter.b for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:54 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:45:54.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:54.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:54.397 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:54 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:45:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:55 smithi184 ceph-mon[33513]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:55 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:56.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:45:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:45:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:55 smithi174 ceph-mon[29878]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:55 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:56.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:55 smithi174 ceph-mon[39523]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:45:56.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:55 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:57.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:45:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:45:57.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:57.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:57.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:57.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:57.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:57.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:57 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:45:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:57 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:57 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[29878]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[39523]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:58.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:58.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:45:58.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:45:58.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:58 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:58 smithi184 ceph-mon[33513]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:45:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:58 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:58 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:58 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:45:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:58 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:45:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:58 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:45:59.702 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:59 smithi174 ceph-mon[29878]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T06:45:59.703 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:45:59 smithi174 ceph-mon[29878]: Reconfiguring daemon alertmanager.a on smithi174 2024-02-10T06:45:59.703 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:59 smithi174 ceph-mon[39523]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T06:45:59.703 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:45:59 smithi174 ceph-mon[39523]: Reconfiguring daemon alertmanager.a on smithi174 2024-02-10T06:46:00.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:59 smithi184 ceph-mon[33513]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T06:46:00.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:45:59 smithi184 ceph-mon[33513]: Reconfiguring daemon alertmanager.a on smithi174 2024-02-10T06:46:00.704 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:00 smithi174 ceph-mon[39523]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:00.704 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:00 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:00 smithi174 ceph-mon[29878]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:00.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:00 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:01.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:00 smithi184 ceph-mon[33513]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:01.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:00 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:02.706 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:02 smithi174 ceph-mon[29878]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:02.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:02 smithi174 ceph-mon[39523]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:03.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:02 smithi184 ceph-mon[33513]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:04.152 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:03 smithi174 ceph-mon[29878]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:04.152 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:03 smithi174 ceph-mon[39523]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:04.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:03 smithi184 ceph-mon[33513]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:05.440 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:05 smithi174 systemd[1]: Stopping Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:46:05.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:05 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[77135]: ts=2024-02-10T06:46:05.439Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-10T06:46:05.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:05 smithi174 podman[78741]: 2024-02-10 06:46:05.469955129 +0000 UTC m=+0.158330621 container died b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:06.228 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:05 smithi184 ceph-mon[33513]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:06.229 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:46:05 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:46:06.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:05 smithi174 ceph-mon[29878]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:06.239 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 podman[78741]: 2024-02-10 06:46:06.094779589 +0000 UTC m=+0.783155080 container cleanup b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:06.239 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 bash[78741]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T06:46:06.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:05 smithi174 ceph-mon[39523]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:06.699 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 podman[78760]: 2024-02-10 06:46:06.329149469 +0000 UTC m=+0.857597891 container remove b5308d7201893b3c45d3842b6a9c1053742e261af87948c0632ec0389a9272de (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:06.699 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 podman[78760]: 2024-02-10 06:46:06.40497963 +0000 UTC m=+0.933428065 volume remove 3a1e44e1e40dee22e8e5092375f97de02266697e56c057ffaa1d034d4667046e 2024-02-10T06:46:06.962 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:06 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:06.962 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-10T06:46:06.962 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 systemd[1]: Stopped Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:46:06.962 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@alertmanager.a.service: Consumed 1.003s CPU time. 2024-02-10T06:46:06.963 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 systemd[1]: Starting Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:46:06.963 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 podman[78848]: 2024-02-10 06:46:06.849539227 +0000 UTC m=+0.026685073 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-10T06:46:06.963 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:06 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:07.238 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:06 smithi174 podman[78848]: 2024-02-10 06:46:06.964044372 +0000 UTC m=+0.141190227 volume create 30ecf2cc03fbc5a696e7f095a4db779e4f82c6f44718e2b6a03d99c48aa5600d 2024-02-10T06:46:07.239 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 podman[78848]: 2024-02-10 06:46:07.064479427 +0000 UTC m=+0.241625281 container create b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:07.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:06 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 podman[78848]: 2024-02-10 06:46:07.358801772 +0000 UTC m=+0.535947627 container init b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 podman[78848]: 2024-02-10 06:46:07.366433197 +0000 UTC m=+0.543579051 container start b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.381Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.381Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.383Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.174 port=9094 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.384Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.409Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T06:46:07.638 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.409Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T06:46:07.639 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.414Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-10T06:46:07.639 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:07.414Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-10T06:46:07.639 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 bash[78848]: b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 2024-02-10T06:46:07.639 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:07 smithi174 systemd[1]: Started Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:46:07.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:46:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[29878]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[29878]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[39523]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:08 smithi174 ceph-mon[39523]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T06:46:08.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:08 smithi184 ceph-mon[33513]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:08.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:08 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:08 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:08 smithi184 ceph-mon[33513]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T06:46:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:09 smithi184 ceph-mon[33513]: Reconfiguring daemon prometheus.a on smithi184 2024-02-10T06:46:09.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:09 smithi174 ceph-mon[29878]: Reconfiguring daemon prometheus.a on smithi184 2024-02-10T06:46:09.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:09 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:09.385Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000628308s 2024-02-10T06:46:09.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:09 smithi174 ceph-mon[39523]: Reconfiguring daemon prometheus.a on smithi184 2024-02-10T06:46:10.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:10 smithi184 ceph-mon[33513]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:10.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:10 smithi174 ceph-mon[29878]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:10.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:10 smithi174 ceph-mon[39523]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:12 smithi184 ceph-mon[33513]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:12 smithi174 ceph-mon[29878]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:12.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:12 smithi174 ceph-mon[39523]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:13 smithi174 ceph-mon[29878]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:13 smithi174 ceph-mon[39523]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:13 smithi184 ceph-mon[33513]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:16.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:15 smithi184 ceph-mon[33513]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:16.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:15 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:16.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:46:15 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:46:16.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:15 smithi174 ceph-mon[29878]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:16.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:15 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:16.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:15 smithi174 ceph-mon[39523]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:16.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:15 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:16.864 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 systemd[1]: Stopping Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:46:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T06:46:17.123 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-10T06:46:17.124 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-10T06:46:17.124 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T06:46:17.124 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-10T06:46:17.124 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[64287]: ts=2024-02-10T06:46:16.864Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-10T06:46:17.124 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:16 smithi184 podman[67671]: 2024-02-10 06:46:16.874589656 +0000 UTC m=+0.138434634 container died aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:17.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:16 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:16 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:17.520 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 podman[67671]: 2024-02-10 06:46:17.303138866 +0000 UTC m=+0.566983851 container cleanup aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:17.521 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 bash[67671]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T06:46:17.738 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 06:46:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T06:46:17.388Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003488504s 2024-02-10T06:46:17.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 podman[67691]: 2024-02-10 06:46:17.520848002 +0000 UTC m=+0.648501018 container remove aac6f60a968c95eec4b7260f761b03f26e821dfc831647f97a853f6380f5a331 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:17.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-10T06:46:17.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 systemd[1]: Stopped Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:46:17.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@prometheus.a.service: Consumed 1.170s CPU time. 2024-02-10T06:46:17.873 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 systemd[1]: Starting Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T06:46:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:17 smithi174 ceph-mon[29878]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:17 smithi174 ceph-mon[39523]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:18.249 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:17 smithi184 ceph-mon[33513]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:18.249 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:17 smithi184 podman[67790]: 2024-02-10 06:46:17.884100268 +0000 UTC m=+0.018960667 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-10T06:46:18.249 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 podman[67790]: 2024-02-10 06:46:18.013657277 +0000 UTC m=+0.148517741 container create d0b9bb860aa0848b72014550e31ff762bfc5d3327d1712f55e462618f102e782 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 podman[67790]: 2024-02-10 06:46:18.249506978 +0000 UTC m=+0.384367386 container init d0b9bb860aa0848b72014550e31ff762bfc5d3327d1712f55e462618f102e782 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 podman[67790]: 2024-02-10 06:46:18.255044536 +0000 UTC m=+0.389904933 container start d0b9bb860aa0848b72014550e31ff762bfc5d3327d1712f55e462618f102e782 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.279Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.279Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.279Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi184 (none))" 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.279Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.279Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T06:46:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.281Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.281Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.282Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.282Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.284Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.284Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.737µs 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.284Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.285Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.285Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.285Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=20.56µs wal_replay_duration=727.346µs wbl_replay_duration=88ns total_replay_duration=766.445µs 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.286Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.286Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.286Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.304Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.203279ms db_storage=801ns remote_storage=1.081µs web_handler=338ns query_engine=501ns scrape=400.389µs scrape_sd=92.99µs notify=15.879µs notify_sd=21.096µs rules=17.459726ms tracing=5.94µs 2024-02-10T06:46:18.624 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.304Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-10T06:46:18.625 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T06:46:18.304Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-10T06:46:18.625 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 bash[67790]: d0b9bb860aa0848b72014550e31ff762bfc5d3327d1712f55e462618f102e782 2024-02-10T06:46:18.625 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 06:46:18 smithi184 systemd[1]: Started Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STOPPING 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STOPPED 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STARTING 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Serving on http://:::9283 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STARTED 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STOPPING 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STOPPED 2024-02-10T06:46:18.739 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STARTING 2024-02-10T06:46:18.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Serving on http://:::9283 2024-02-10T06:46:18.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STARTED 2024-02-10T06:46:18.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STOPPING 2024-02-10T06:46:18.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T06:46:18.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STOPPED 2024-02-10T06:46:18.740 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STARTING 2024-02-10T06:46:19.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Serving on http://:::9283 2024-02-10T06:46:19.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:06:46:18] ENGINE Bus STARTED 2024-02-10T06:46:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T06:46:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi174.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi174.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T06:46:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T06:46:19.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T06:46:19.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:19 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:46:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T06:46:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T06:46:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi174.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi174.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T06:46:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi174.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi174.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T06:46:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi184.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi184.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:19.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:19 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:46:20.481 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:20 smithi184 ceph-mon[33513]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:20.723 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:20 smithi174 ceph-mon[29878]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:20.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:20 smithi174 ceph-mon[39523]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:22.732 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:22 smithi184 ceph-mon[33513]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:22.733 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:22.733 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:22.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:22 smithi174 ceph-mon[29878]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:22.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:22 smithi174 ceph-mon[39523]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[29878]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[39523]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:46:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:46:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:23 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:23 smithi184 ceph-mon[33513]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:23 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:23 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:23 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:46:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:23 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:46:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:23 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:25.635 INFO:tasks.workunit.client.0.smithi174.stderr:Note: switching to '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'. 2024-02-10T06:46:25.635 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.635 INFO:tasks.workunit.client.0.smithi174.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-10T06:46:25.635 INFO:tasks.workunit.client.0.smithi174.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr:state without impacting any branches by switching back to a branch. 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: git switch -c 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr:Or undo this operation with: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: git switch - 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr: 2024-02-10T06:46:25.636 INFO:tasks.workunit.client.0.smithi174.stderr:HEAD is now at 290212a1bfd Merge pull request #53521 from xxhdx1985126/wip-62857 2024-02-10T06:46:25.640 DEBUG:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-10T06:46:25.663 INFO:tasks.workunit.client.0.smithi174.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-10T06:46:25.665 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-10T06:46:25.665 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-10T06:46:26.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:25 smithi174 ceph-mon[29878]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:25 smithi174 ceph-mon[39523]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:25 smithi184 ceph-mon[33513]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:26.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:46:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:46:26.447 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-10T06:46:26.516 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-10T06:46:26.555 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-10T06:46:26.556 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-10T06:46:26.556 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-10T06:46:26.596 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-10T06:46:26.599 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-02-10T06:46:26.599 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-10T06:46:26.655 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-10T06:46:26.656 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-10T06:46:26.657 DEBUG:teuthology.orchestra.run.smithi174:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-10T06:46:26.765 INFO:tasks.workunit.client.0.smithi174.stderr:+ parallel=1 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' '' = --serial ']' 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ crimson=0 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' '' = --crimson ']' 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ color= 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -t 1 ']' 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-02-10T06:46:26.766 INFO:tasks.workunit.client.0.smithi174.stderr:+ declare -A pids 2024-02-10T06:46:26.767 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.767 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.767 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_aio 2024-02-10T06:46:26.767 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_aio' 2024-02-10T06:46:26.767 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_aio 2024-02-10T06:46:26.767 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.768 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_aio 2024-02-10T06:46:26.768 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79622 2024-02-10T06:46:26.768 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_aio on pid 79622' 2024-02-10T06:46:26.768 INFO:tasks.workunit.client.0.smithi174.stdout:test api_aio on pid 79622 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79622 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_aio_pp 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_aio_pp' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_aio_pp 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.771 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.772 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_aio_pp 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79629 2024-02-10T06:46:26.773 INFO:tasks.workunit.client.0.smithi174.stdout:test api_aio_pp on pid 79629 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_aio_pp on pid 79629' 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79629 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_io 2024-02-10T06:46:26.776 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_io' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_io 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.777 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.778 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.779 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.780 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_io 2024-02-10T06:46:26.780 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79638 2024-02-10T06:46:26.780 INFO:tasks.workunit.client.0.smithi174.stdout:test api_io on pid 79638 2024-02-10T06:46:26.784 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_io on pid 79638' 2024-02-10T06:46:26.784 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79638 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_io_pp 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_io_pp' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_io_pp 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.785 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.786 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.787 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.788 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_io_pp 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79649 2024-02-10T06:46:26.789 INFO:tasks.workunit.client.0.smithi174.stdout:test api_io_pp on pid 79649 2024-02-10T06:46:26.795 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_io_pp on pid 79649' 2024-02-10T06:46:26.795 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79649 2024-02-10T06:46:26.795 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.795 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.795 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_asio 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_asio' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_asio 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.796 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.797 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.798 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.799 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.800 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.801 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_asio 2024-02-10T06:46:26.802 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79664 2024-02-10T06:46:26.802 INFO:tasks.workunit.client.0.smithi174.stdout:test api_asio on pid 79664 2024-02-10T06:46:26.807 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_asio on pid 79664' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79664 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_list 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_list' 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_list 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.809 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.810 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.811 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.812 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.813 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.814 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.815 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.816 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79622/exe 2024-02-10T06:46:26.816 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_list 2024-02-10T06:46:26.816 INFO:tasks.workunit.client.0.smithi174.stdout:test api_list on pid 79682 2024-02-10T06:46:26.819 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79682 2024-02-10T06:46:26.819 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_list on pid 79682' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79682 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_lock 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_lock' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.820 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.821 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_lock 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.822 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.823 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.824 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.825 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79629/exe 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_aio: /' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.826 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.827 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.828 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.828 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.828 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.828 INFO:tasks.workunit.client.0.smithi174.stdout:test api_lock on pid 79702 2024-02-10T06:46:26.830 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.830 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_lock 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79702 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_lock on pid 79702' 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79702 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.831 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_lock_pp 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_lock_pp' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.832 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.833 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79638/exe 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.834 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_lock_pp 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.835 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.836 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.837 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.838 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.839 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.839 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.839 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.839 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.839 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.839 INFO:tasks.workunit.client.0.smithi174.stdout:test api_lock_pp on pid 79724 2024-02-10T06:46:26.842 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_lock_pp 2024-02-10T06:46:26.842 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79724 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_lock_pp on pid 79724' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79724 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_io.log 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_io: /' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.843 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.844 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_misc 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_misc' 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_misc 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.845 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.846 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.847 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.848 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79649/exe 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.849 INFO:tasks.workunit.client.0.smithi174.stdout:test api_misc on pid 79740 2024-02-10T06:46:26.852 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_misc 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79740 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_misc on pid 79740' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79740 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.853 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.854 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_misc_pp 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.855 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_misc_pp' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.856 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.857 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79664/exe 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.858 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.859 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.860 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.861 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.862 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.863 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.864 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_misc_pp 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_misc_pp 2024-02-10T06:46:26.865 INFO:tasks.workunit.client.0.smithi174.stdout:test api_misc_pp on pid 79768 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79768 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_misc_pp on pid 79768' 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79768 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_tier_pp 2024-02-10T06:46:26.869 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_tier_pp' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_asio: /' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.870 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_tier_pp 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.871 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79682/exe 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.872 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.873 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_list: /' 2024-02-10T06:46:26.874 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2024-02-10T06:46:26.875 INFO:tasks.workunit.client.0.smithi174.stdout:test api_tier_pp on pid 79782 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_tier_pp 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79782 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_tier_pp on pid 79782' 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79782 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.880 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.881 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_pool 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_pool' 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_list.log 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.882 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.883 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.884 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.885 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.886 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.887 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.888 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.889 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.889 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.889 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_pool 2024-02-10T06:46:26.889 INFO:tasks.workunit.client.0.smithi174.stdout:test api_pool on pid 79819 2024-02-10T06:46:26.893 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_pool 2024-02-10T06:46:26.893 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79702/exe 2024-02-10T06:46:26.893 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79819 2024-02-10T06:46:26.893 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_pool on pid 79819' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79819 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.894 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_snapshots 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.895 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_snapshots' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.896 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.897 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.898 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_snapshots 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.899 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.900 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.901 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.902 INFO:tasks.workunit.client.0.smithi174.stdout:test api_snapshots on pid 79880 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_snapshots 2024-02-10T06:46:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79880 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_snapshots on pid 79880' 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79880 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_lock: /' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_snapshots_pp 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_snapshots_pp' 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.911 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_snapshots_pp 2024-02-10T06:46:26.912 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79768/exe 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79724/exe 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.913 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_snapshots_pp 2024-02-10T06:46:26.914 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.914 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.914 INFO:tasks.workunit.client.0.smithi174.stdout:test api_snapshots_pp on pid 79920 2024-02-10T06:46:26.920 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79920 2024-02-10T06:46:26.920 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_snapshots_pp on pid 79920' 2024-02-10T06:46:26.920 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79920 2024-02-10T06:46:26.920 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.920 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.921 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-10T06:46:26.922 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.923 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.924 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.925 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.926 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.927 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_stat 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.928 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.929 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.930 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.931 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_stat' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.932 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.933 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79782/exe 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.934 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79740/exe 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.935 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.936 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.937 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.938 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.939 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.940 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79880/exe 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_misc: /' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.941 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.942 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_stat 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.943 INFO:tasks.workunit.client.0.smithi174.stdout:test api_stat on pid 79982 2024-02-10T06:46:26.947 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_stat 2024-02-10T06:46:26.947 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79982 2024-02-10T06:46:26.947 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_stat on pid 79982' 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=79982 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_stat_pp 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_stat_pp' 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79920/exe 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.948 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.949 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.950 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.951 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-10T06:46:26.952 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-10T06:46:26.953 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.953 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_stat_pp 2024-02-10T06:46:26.953 INFO:tasks.workunit.client.0.smithi174.stdout:test api_stat_pp on pid 80007 2024-02-10T06:46:26.956 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_stat_pp 2024-02-10T06:46:26.956 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79819/exe 2024-02-10T06:46:26.956 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80007 2024-02-10T06:46:26.956 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_stat_pp on pid 80007' 2024-02-10T06:46:26.956 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80007 2024-02-10T06:46:26.956 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_watch_notify 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_watch_notify' 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:26.957 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.958 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.959 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.960 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_watch_notify 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_pool: /' 2024-02-10T06:46:26.961 INFO:tasks.workunit.client.0.smithi174.stdout:test api_watch_notify on pid 80043 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_watch_notify 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80043 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_watch_notify on pid 80043' 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80043 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.962 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.963 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_watch_notify_pp 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_watch_notify_pp' 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_watch_notify_pp 2024-02-10T06:46:26.964 INFO:tasks.workunit.client.0.smithi174.stdout:test api_watch_notify_pp on pid 80065 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_watch_notify_pp 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80065 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_watch_notify_pp on pid 80065' 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80065 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.968 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_cmd 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_cmd' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.969 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.970 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_cmd 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.971 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.972 INFO:tasks.workunit.client.0.smithi174.stdout:test api_cmd on pid 80083 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_cmd 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80083 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_cmd on pid 80083' 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80083 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.973 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_cmd_pp 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_cmd_pp' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.974 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.975 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.976 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_cmd_pp 2024-02-10T06:46:26.977 INFO:tasks.workunit.client.0.smithi174.stdout:test api_cmd_pp on pid 80099 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_cmd_pp 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80099 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_cmd_pp on pid 80099' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80099 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.982 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.983 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.984 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.985 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.986 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.987 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.988 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.989 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.990 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.991 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.992 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.993 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:26.994 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:26.995 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_service 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_service' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:26.996 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.997 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:26.998 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:26.999 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/79982/exe 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_service 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.000 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.001 INFO:tasks.workunit.client.0.smithi174.stdout:test api_service on pid 80152 2024-02-10T06:46:27.004 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_service 2024-02-10T06:46:27.004 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80152 2024-02-10T06:46:27.004 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_service on pid 80152' 2024-02-10T06:46:27.004 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80152 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_service_pp 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_service_pp' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.005 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80065/exe 2024-02-10T06:46:27.006 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.007 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.008 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_stat: /' 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.009 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.010 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80007/exe 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.011 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.012 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.013 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.014 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_service_pp 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.015 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-10T06:46:27.016 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80083/exe 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.017 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.018 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.019 INFO:tasks.workunit.client.0.smithi174.stdout:test api_service_pp on pid 80200 2024-02-10T06:46:27.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_service_pp 2024-02-10T06:46:27.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80200 2024-02-10T06:46:27.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_service_pp on pid 80200' 2024-02-10T06:46:27.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80200 2024-02-10T06:46:27.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.024 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80043/exe 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_cmd: /' 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-10T06:46:27.025 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_c_write_operations 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_c_write_operations' 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.026 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.027 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.028 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.029 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.030 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.031 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_c_write_operations 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.032 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.033 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_c_write_operations 2024-02-10T06:46:27.034 INFO:tasks.workunit.client.0.smithi174.stdout:test api_c_write_operations on pid 80224 2024-02-10T06:46:27.037 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80224 2024-02-10T06:46:27.037 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_c_write_operations on pid 80224' 2024-02-10T06:46:27.037 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80224 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.038 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.039 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80152/exe 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_c_read_operations 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_c_read_operations' 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.040 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_service: /' 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80099/exe 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_service.log 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.041 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.042 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_c_read_operations 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.043 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.044 INFO:tasks.workunit.client.0.smithi174.stdout:test api_c_read_operations on pid 80274 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_c_read_operations 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80274 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_c_read_operations on pid 80274' 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80274 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.048 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.049 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s api_cls_remote_reads 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.050 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' api_cls_remote_reads' 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.051 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.052 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.053 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80200/exe 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo api_cls_remote_reads 2024-02-10T06:46:27.054 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.055 INFO:tasks.workunit.client.0.smithi174.stdout:test api_cls_remote_reads on pid 80332 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=api_cls_remote_reads 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.058 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80332 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test api_cls_remote_reads on pid 80332' 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80332 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s list_parallel 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' list_parallel' 2024-02-10T06:46:27.059 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.060 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.061 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.062 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.063 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo list_parallel 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.064 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.065 INFO:tasks.workunit.client.0.smithi174.stdout:test list_parallel on pid 80349 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=list_parallel 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80349 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test list_parallel on pid 80349' 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80349 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s open_pools_parallel 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' open_pools_parallel' 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.066 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.067 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.068 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.068 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo open_pools_parallel 2024-02-10T06:46:27.068 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.068 INFO:tasks.workunit.client.0.smithi174.stdout:test open_pools_parallel on pid 80362 2024-02-10T06:46:27.069 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=open_pools_parallel 2024-02-10T06:46:27.069 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80362 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test open_pools_parallel on pid 80362' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80362 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s delete_pools_parallel 2024-02-10T06:46:27.070 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' delete_pools_parallel' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.071 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.072 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo delete_pools_parallel 2024-02-10T06:46:27.073 INFO:tasks.workunit.client.0.smithi174.stdout:test delete_pools_parallel on pid 80379 2024-02-10T06:46:27.075 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=delete_pools_parallel 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80379 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test delete_pools_parallel on pid 80379' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80379 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s cls 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' cls' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.076 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.077 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.078 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo cls 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.079 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.080 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.080 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=cls 2024-02-10T06:46:27.080 INFO:tasks.workunit.client.0.smithi174.stdout:test cls on pid 80397 2024-02-10T06:46:27.081 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80397 2024-02-10T06:46:27.081 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test cls on pid 80397' 2024-02-10T06:46:27.081 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80397 2024-02-10T06:46:27.081 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.081 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s cmd 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' cmd' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.082 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.083 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80274/exe 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.084 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo cmd 2024-02-10T06:46:27.086 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-02-10T06:46:27.086 INFO:tasks.workunit.client.0.smithi174.stdout:test cmd on pid 80407 2024-02-10T06:46:27.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=cmd 2024-02-10T06:46:27.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80407 2024-02-10T06:46:27.091 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test cmd on pid 80407' 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80407 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.092 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s handler_error 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' handler_error' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.093 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.094 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.095 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo handler_error 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.096 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.097 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.098 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.099 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.099 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.099 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.099 INFO:tasks.workunit.client.0.smithi174.stdout:test handler_error on pid 80427 2024-02-10T06:46:27.101 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.101 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.101 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.101 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=handler_error 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.102 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.103 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80427 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test handler_error on pid 80427' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80427 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s io 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.104 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' io' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.105 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo io 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.106 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.107 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=io 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80442 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.108 INFO:tasks.workunit.client.0.smithi174.stdout:test io on pid 80442 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test io on pid 80442' 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80442 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.112 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.113 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.114 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.115 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.116 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80332/exe 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.117 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s ec_io 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.118 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' ec_io' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.119 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80224/exe 2024-02-10T06:46:27.120 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.121 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.122 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.123 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.124 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.125 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo ec_io 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.126 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.127 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80362/exe 2024-02-10T06:46:27.128 INFO:tasks.workunit.client.0.smithi174.stdout:test ec_io on pid 80513 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=ec_io 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80513 2024-02-10T06:46:27.135 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test ec_io on pid 80513' 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80513 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.136 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.137 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.138 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.139 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.140 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2024-02-10T06:46:27.141 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.142 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s list 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' list' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.143 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo list 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.144 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80397/exe 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.145 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.146 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.147 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_cls.log 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.148 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.149 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.150 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_cls 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.151 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.152 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.153 INFO:tasks.workunit.client.0.smithi174.stdout:test list on pid 80568 2024-02-10T06:46:27.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=list 2024-02-10T06:46:27.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80568 2024-02-10T06:46:27.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test list on pid 80568' 2024-02-10T06:46:27.160 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80568 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80349/exe 2024-02-10T06:46:27.161 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s ec_list 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.162 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.163 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.164 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' ec_list' 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ cls: /' 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.165 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.166 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80442/exe 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.167 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.168 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.169 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_io 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ io: /' 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_io.log 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.170 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.171 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80379/exe 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo ec_list 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.172 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.173 INFO:tasks.workunit.client.0.smithi174.stdout:test ec_list on pid 80608 2024-02-10T06:46:27.178 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=ec_list 2024-02-10T06:46:27.178 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80608 2024-02-10T06:46:27.178 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test ec_list on pid 80608' 2024-02-10T06:46:27.178 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80608 2024-02-10T06:46:27.178 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ list_parallel: /' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s misc 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80427/exe 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' misc' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.179 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.180 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.181 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.182 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.183 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.184 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_handler_error 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.185 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_handler_error.log 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ handler_error: /' 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo misc 2024-02-10T06:46:27.186 INFO:tasks.workunit.client.0.smithi174.stdout:test misc on pid 80666 2024-02-10T06:46:27.188 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=misc 2024-02-10T06:46:27.188 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80666 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test misc on pid 80666' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80666 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.189 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s pool 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' pool' 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80513/exe 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo pool 2024-02-10T06:46:27.190 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.191 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.192 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.193 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.194 INFO:tasks.workunit.client.0.smithi174.stdout:test pool on pid 80719 2024-02-10T06:46:27.197 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=pool 2024-02-10T06:46:27.197 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80719 2024-02-10T06:46:27.197 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test pool on pid 80719' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80719 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_ec_io 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.198 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s read_operations 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.199 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' read_operations' 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.200 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.201 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.202 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.202 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_ec_io.log 2024-02-10T06:46:27.202 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo read_operations 2024-02-10T06:46:27.202 INFO:tasks.workunit.client.0.smithi174.stdout:test read_operations on pid 80741 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=read_operations 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80741 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test read_operations on pid 80741' 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80741 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.203 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ ec_io: /' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s snapshots 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' snapshots' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.204 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.205 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.206 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.206 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.206 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.207 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo snapshots 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.208 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.209 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.210 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=snapshots 2024-02-10T06:46:27.211 INFO:tasks.workunit.client.0.smithi174.stdout:test snapshots on pid 80762 2024-02-10T06:46:27.213 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80762 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test snapshots on pid 80762' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80762 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s watch_notify 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' watch_notify' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.214 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.215 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.216 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.217 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80407/exe 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.218 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.219 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo watch_notify 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_cmd.log 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=watch_notify 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ cmd: /' 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80781 2024-02-10T06:46:27.220 INFO:tasks.workunit.client.0.smithi174.stdout:test watch_notify on pid 80781 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test watch_notify on pid 80781' 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80781 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.223 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.224 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_cmd 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ printf %25s write_operations 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:+ r=' write_operations' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.225 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ echo write_operations 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.226 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.227 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.228 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.229 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.230 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80608/exe 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.232 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.233 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_ec_list 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.234 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.235 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_ec_list.log 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.236 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.237 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.238 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ ec_list: /' 2024-02-10T06:46:27.239 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.240 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.241 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.242 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.243 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.244 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.244 INFO:tasks.workunit.client.0.smithi174.stderr:++ awk '{print $1}' 2024-02-10T06:46:27.244 INFO:tasks.workunit.client.0.smithi174.stdout:test write_operations on pid 80845 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ ff=write_operations 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80845 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo 'test write_operations on pid 80845' 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ pids[$f]=80845 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ ret=0 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79782 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79782 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-02-10T06:46:27.248 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80666/exe 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.249 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+ . /etc/bashrc 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -z '' ']' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ BASHRCSOURCED=Y 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ shopt -q login_shell 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.250 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:+++ umask 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.251 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ SHELL=/bin/bash 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T06:46:27.252 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80568/exe 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.253 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80762/exe 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.254 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_misc 2024-02-10T06:46:27.257 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.257 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ misc: /' 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_misc.log 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80719/exe 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ list: /' 2024-02-10T06:46:27.258 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.259 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_list.log 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_snapshots 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_list 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_snapshots.log 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T06:46:27.260 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:+++ return 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export S_COLORS=auto 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:+++ S_COLORS=auto 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.261 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ snapshots: /' 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.262 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.263 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.264 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.264 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.264 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.264 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_pool 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.268 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.269 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ pool: /' 2024-02-10T06:46:27.269 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_pool.log 2024-02-10T06:46:27.272 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T06:46:27.273 INFO:tasks.workunit.client.0.smithi174.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.276 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80741/exe 2024-02-10T06:46:27.277 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.278 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T06:46:27.278 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T06:46:27.278 INFO:tasks.workunit.client.0.smithi174.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T06:46:27.278 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T06:46:27.279 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.280 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ read_operations: /' 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.281 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.282 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.282 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.282 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.282 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.283 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.284 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.284 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.284 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80845/exe 2024-02-10T06:46:27.284 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_read_operations 2024-02-10T06:46:27.284 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.284 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_read_operations.log 2024-02-10T06:46:27.285 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/locale 2024-02-10T06:46:27.286 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_write_operations.log 2024-02-10T06:46:27.287 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_write_operations 2024-02-10T06:46:27.292 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ write_operations: /' 2024-02-10T06:46:27.294 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.294 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T06:46:27.294 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.294 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T06:46:27.294 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T06:46:27.294 INFO:tasks.workunit.client.0.smithi174.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T06:46:27.296 INFO:tasks.workunit.client.0.smithi174.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T06:46:27.296 INFO:tasks.workunit.client.0.smithi174.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:++++ LANG=en_US.UTF-8 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ LANG=en_US.UTF-8 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ unset LANG_backup config 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n '' ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' dumb = linux ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/less.sh 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T06:46:27.297 INFO:tasks.workunit.client.0.smithi174.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T06:46:27.298 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T06:46:27.298 INFO:tasks.workunit.client.0.smithi174.stderr:++ '[' '' ']' 2024-02-10T06:46:27.298 INFO:tasks.workunit.client.0.smithi174.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T06:46:27.298 INFO:tasks.workunit.client.0.smithi174.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T06:46:27.298 INFO:tasks.workunit.client.0.smithi174.stderr:+++++ readlink /proc/80781/exe 2024-02-10T06:46:27.298 INFO:tasks.workunit.client.0.smithi174.stderr:++++ basename /usr/bin/bash 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_declare='declare -f' 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+++ which_opt=-f 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export which_declare 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+++ export -f which 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset i 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:++ unset -f pathmunge 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+ export PATH 2024-02-10T06:46:27.300 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T06:46:27.301 INFO:tasks.workunit.client.0.smithi174.stderr:+ unset rc 2024-02-10T06:46:27.301 INFO:tasks.workunit.client.0.smithi174.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-02-10T06:46:27.301 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph_test_neorados_watch_notify 2024-02-10T06:46:27.301 INFO:tasks.workunit.client.0.smithi174.stderr:+ sed 's/^/ watch_notify: /' 2024-02-10T06:46:27.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:27.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:28.071 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:46:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1370761282' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[29878]: osdmap e72: 8 total, 8 up, 8 in 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:28.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1370761282' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T06:46:28.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:28.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:28.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:28 smithi174 ceph-mon[39523]: osdmap e72: 8 total, 8 up, 8 in 2024-02-10T06:46:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1370761282' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T06:46:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:28 smithi184 ceph-mon[33513]: osdmap e72: 8 total, 8 up, 8 in 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/565882353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi174-79693-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3845713657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi174-79705-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1944077256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/643158426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3061234830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2806469158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[39523]: osdmap e73: 8 total, 8 up, 8 in 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/565882353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi174-79693-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3845713657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi174-79705-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1944077256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/643158426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3061234830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2806469158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:29 smithi174 ceph-mon[29878]: osdmap e73: 8 total, 8 up, 8 in 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/565882353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi174-79693-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3845713657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi174-79705-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1944077256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/643158426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3061234830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2806469158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:29 smithi184 ceph-mon[33513]: osdmap e73: 8 total, 8 up, 8 in 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[29878]: pgmap v57: 356 pgs: 224 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[29878]: Health check failed: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[39523]: pgmap v57: 356 pgs: 224 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[39523]: Health check failed: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:33 smithi184 ceph-mon[33513]: pgmap v57: 356 pgs: 224 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:46:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:33 smithi184 ceph-mon[33513]: Health check failed: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/565882353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi174-79693-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3845713657' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi174-79705-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1074687794' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/590887014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3696160538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: pgmap v59: 516 pgs: 307 creating+peering, 158 active+clean, 51 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: osdmap e74: 8 total, 8 up, 8 in 2024-02-10T06:46:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi174-79779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3426730933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4253780278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4210616504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/148224638' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/672364339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/565882353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi174-79693-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3845713657' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi174-79705-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1074687794' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/590887014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3696160538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: pgmap v59: 516 pgs: 307 creating+peering, 158 active+clean, 51 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: osdmap e74: 8 total, 8 up, 8 in 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi174-79779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3426730933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4253780278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4210616504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/148224638' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/672364339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2210806706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/17717523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3126925877' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: osdmap e75: 8 total, 8 up, 8 in 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/989630150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi174-79986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2210806706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/17717523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3126925877' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:35.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: osdmap e75: 8 total, 8 up, 8 in 2024-02-10T06:46:35.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:35.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:35.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/989630150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi174-79986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/565882353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi174-79693-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3845713657' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi174-79705-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1074687794' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/590887014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3696160538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: pgmap v59: 516 pgs: 307 creating+peering, 158 active+clean, 51 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: osdmap e74: 8 total, 8 up, 8 in 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi174-79779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi174-79860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi174-80160-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3426730933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4253780278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4210616504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/148224638' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/672364339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2210806706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/17717523' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:35.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3126925877' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:35.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: osdmap e75: 8 total, 8 up, 8 in 2024-02-10T06:46:35.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:35.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:35.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/989630150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi174-79986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.287 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:46:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: pgmap v60: 516 pgs: 307 creating+peering, 158 active+clean, 51 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: pgmap v62: 1060 pgs: 307 creating+peering, 158 active+clean, 595 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.288 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi174-79779-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/989630150' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi174-79986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: osdmap e76: 8 total, 8 up, 8 in 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.289 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.290 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:36.290 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: pgmap v60: 516 pgs: 307 creating+peering, 158 active+clean, 51 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: pgmap v62: 1060 pgs: 307 creating+peering, 158 active+clean, 595 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-10T06:46:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi174-79779-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/989630150' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi174-79986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: osdmap e76: 8 total, 8 up, 8 in 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: pgmap v60: 516 pgs: 307 creating+peering, 158 active+clean, 51 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: pgmap v62: 1060 pgs: 307 creating+peering, 158 active+clean, 595 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi174-79934-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi174-79779-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi174-80818-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/989630150' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi174-79986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: osdmap e76: 8 total, 8 up, 8 in 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:36.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:36.587 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-10T06:46:36.587 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] Global test environment set-up. 2024-02-10T06:46:36.587 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-10T06:46:36.587 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-10T06:46:36.587 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (21 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (7 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (3 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (9 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (4 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-02-10T06:46:36.588 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (2 ms) 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (2 ms) 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3 ms) 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (4 ms) 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] 12 tests from AsioRados (59 ms total) 2024-02-10T06:46:36.589 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: 2024-02-10T06:46:36.590 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [----------] Global test environment tear-down 2024-02-10T06:46:36.590 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (9734 ms total) 2024-02-10T06:46:36.590 INFO:tasks.workunit.client.0.smithi174.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] Global test environment set-up. 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjects (670 ms) 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (33 ms) 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo2,foo3 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo2 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo3 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo4,foo5 2024-02-10T06:46:36.831 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo4 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo5 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo6,foo7 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo7 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo6 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo4 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo5 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo7 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo6 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo1 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo1 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo2 2024-02-10T06:46:36.832 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo3 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (72 ms) 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 1 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 10 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 13 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 7 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 14 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 0 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 15 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 11 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 5 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 8 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 6 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 3 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 4 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 12 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 9 0 2024-02-10T06:46:36.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2 0 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (59 ms) 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: x cursor=MIN 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=1 cursor=18:02547ec2:::1:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=10 cursor=18:52ea6a34:::10:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=13 cursor=18:566253c9:::13:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=7 cursor=18:5c6b0b28:::7:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=14 cursor=18:62a1935d:::14:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=0 cursor=18:6cac518f:::0:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=15 cursor=18:863748b0:::15:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=11 cursor=18:89d3ae78:::11:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=5 cursor=18:b29083e3:::5:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=8 cursor=18:bd63b0f1:::8:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=6 cursor=18:c4fdafeb:::6:head 2024-02-10T06:46:36.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=3 cursor=18:cfc208b3:::3:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=4 cursor=18:d83876eb:::4:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=12 cursor=18:de5d7c5f:::12:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=9 cursor=18:e960b815:::9:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > oid=2 cursor=18:f905c69b:::2:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=1 cursor=18:02547ec2:::1:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:02547ec2:::1:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:02547ec2:::1:head -> 1 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=10 cursor=18:52ea6a34:::10:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:52ea6a34:::10:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:52ea6a34:::10:head -> 10 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=13 cursor=18:566253c9:::13:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:566253c9:::13:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:566253c9:::13:head -> 13 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=7 cursor=18:5c6b0b28:::7:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:5c6b0b28:::7:head 2024-02-10T06:46:36.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:5c6b0b28:::7:head -> 7 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=14 cursor=18:62a1935d:::14:head 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:62a1935d:::14:head 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:62a1935d:::14:head -> 14 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=0 cursor=18:6cac518f:::0:head 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:6cac518f:::0:head 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:6cac518f:::0:head -> 0 2024-02-10T06:46:36.906 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=15 cursor=18:863748b0:::15:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:863748b0:::15:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:863748b0:::15:head -> 15 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=11 cursor=18:89d3ae78:::11:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:89d3ae78:::11:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:89d3ae78:::11:head -> 11 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=5 cursor=18:b29083e3:::5:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:b29083e3:::5:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:b29083e3:::5:head -> 5 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=8 cursor=18:bd63b0f1:::8:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:bd63b0f1:::8:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:bd63b0f1:::8:head -> 8 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=6 cursor=18:c4fdafeb:::6:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:c4fdafeb:::6:head 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:c4fdafeb:::6:head -> 6 2024-02-10T06:46:36.907 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=3 cursor=18:cfc208b3:::3:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:cfc208b3:::3:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:cfc208b3:::3:head -> 3 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=4 cursor=18:d83876eb:::4:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:d83876eb:::4:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:d83876eb:::4:head -> 4 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=12 cursor=18:de5d7c5f:::12:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:de5d7c5f:::12:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:de5d7c5f:::12:head -> 12 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=9 cursor=18:e960b815:::9:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:e960b815:::9:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:e960b815:::9:head -> 9 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : oid=2 cursor=18:f905c69b:::2:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:f905c69b:::2:head 2024-02-10T06:46:36.908 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:f905c69b:::2:head -> 2 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:5c6b0b28:::7:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:5c6b0b28:::7:head expected=18:5c6b0b28:::7:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:5c6b0b28:::7:head -> 7 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=7 expected=7 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:52ea6a34:::10:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:52ea6a34:::10:head expected=18:52ea6a34:::10:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:52ea6a34:::10:head -> 10 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=10 expected=10 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:b29083e3:::5:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:b29083e3:::5:head expected=18:b29083e3:::5:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:b29083e3:::5:head -> 5 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=5 expected=5 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:89d3ae78:::11:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:89d3ae78:::11:head expected=18:89d3ae78:::11:head 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:89d3ae78:::11:head -> 11 2024-02-10T06:46:36.909 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=11 expected=11 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:863748b0:::15:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:863748b0:::15:head expected=18:863748b0:::15:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:863748b0:::15:head -> 15 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=15 expected=15 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:6cac518f:::0:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:6cac518f:::0:head expected=18:6cac518f:::0:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:6cac518f:::0:head -> 0 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=0 expected=0 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:cfc208b3:::3:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:cfc208b3:::3:head expected=18:cfc208b3:::3:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:cfc208b3:::3:head -> 3 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=3 expected=3 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:c4fdafeb:::6:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:c4fdafeb:::6:head expected=18:c4fdafeb:::6:head 2024-02-10T06:46:36.910 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:c4fdafeb:::6:head -> 6 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : e list_parallel: process_1_[80623]: starting. 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[80623]: creating pool ceph_test_rados_list_parallel.smithi174-80576 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[80623]: created object 0... 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[80623]: created object 25... 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[80623]: created object 49... 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[80623]: finishing. 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_1_[80623]: shutting down. 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[80628]: starting. 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[80628]: listing objects. 2024-02-10T06:46:37.132 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[80628]: listed object 0... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[80628]: listed object 25... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[80628]: saw 50 objects 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_2_[80628]: shutting down. 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: starting. 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: creating pool ceph_test_rados_list_parallel.smithi174-80576 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: created object 0... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: created object 25... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: created object 49... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: finishing. 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_3_[81717]: shutting down. 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[81718]: starting. 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[81718]: listing objects. 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[81718]: listed object 0... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[81718]: listed object 25... 2024-02-10T06:46:37.133 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[81718]: saw 50 objects 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_4_[81718]: shutting down. 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[81719]: starting. 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[81719]: removed 25 objects... 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[81719]: removed half of the objects 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[81719]: removed 50 objects... 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[81719]: removed 50 objects 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_5_[81719]: shutting down. 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: starting. 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: creating pool ceph_test_rados_list_parallel.smithi174-80576 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: created object 0... 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: created object 25... 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: created object 49... 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: finishing. 2024-02-10T06:46:37.134 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_6_[81767]: shutting down. 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: starting. 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: listing objects. 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: listed object 0... 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: listed object 25... 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: listed object 50... 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: saw 56 objects 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_7_[81768]: shutting down. 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[81769]: starting. 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[81769]: added 25 objects... 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[81769]: added half of the objects 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[81769]: added 50 objects... 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[81769]: added 50 objects 2024-02-10T06:46:37.135 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_8_[81769]: shutting down. 2024-02-10T06:46:37.136 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.136 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: starting. 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: creating pool ceph_test_rados_list_parallel.smithi174-80576 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: created object 0... 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: created object 25... 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: created object 49... 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: finishing. 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_9_[81815]: shutting down. 2024-02-10T06:46:37.310 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: starting. 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: listing objects. 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: listed object 0... 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: listed object 25... 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: listed object 50... 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: listed object 75... 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: saw 98 objects 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_10_[81816]: shutting down. 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[81819]: starting. 2024-02-10T06:46:37.311 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[81819]: removed 25 objects... 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[81819]: removed half of the objects 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[81819]: removed 50 objects... 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[81819]: removed 50 objects 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_13_[81819]: shutting down. 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[81817]: starting. 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[81817]: added 25 objects... 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[81817]: added half of the objects 2024-02-10T06:46:37.312 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[81817]: added 50 objects... 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[81817]: added 50 objects 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_11_[81817]: shutting down. 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[81818]: starting. 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[81818]: added 25 objects... 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[81818]: added half of the objects 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[81818]: added 50 objects... 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[81818]: added 50 objects 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_12_[81818]: shutting down. 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.313 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: starting. 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: creating pool ceph_test_rados_list_parallel.smithi174-80576 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: created object 0... 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: created object 25... 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: created object 49... 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: finishing. 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_14_[81897]: shutting down. 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: starting. 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listing objects. 2024-02-10T06:46:37.314 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listed object 0... 2024-02-10T06:46:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3028716066' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:37.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3028716066' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2265060714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/754509825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: osdmap e77: 8 total, 8 up, 8 in 2024-02-10T06:46:37.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2265060714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/754509825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: osdmap e77: 8 total, 8 up, 8 in 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:37.827 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3028716066' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:37.828 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2265060714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/754509825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: osdmap e77: 8 total, 8 up, 8 in 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:37.829 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listed object 2 handler_error: Running main() from gmock_main.cc 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [----------] Global test environment set-up. 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (10817 ms) 2024-02-10T06:46:38.007 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [----------] 1 test from neocls_handler_error (10817 ms total) 2024-02-10T06:46:38.008 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: 2024-02-10T06:46:38.008 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [----------] Global test environment tear-down 2024-02-10T06:46:38.008 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [==========] 1 test from 1 test suite ran. (10817 ms total) 2024-02-10T06:46:38.008 INFO:tasks.workunit.client.0.smithi174.stdout: handler_error: [ PASSED ] 1 test. 2024-02-10T06:46:38.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-02-10T06:46:38.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-02-10T06:46:38.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (10863 ms) 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (10863 ms total) 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (10863 ms total) 2024-02-10T06:46:38.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (47 ms) 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (435 ms) 2024-02-10T06:46:38.018 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-10T06:46:38.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (10508 ms) 2024-02-10T06:46:38.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (10990 ms total) 2024-02-10T06:46:38.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: 2024-02-10T06:46:38.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-10T06:46:38.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (10990 ms total) 2024-02-10T06:46:38.019 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (51 ms) 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (2152 ms) 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-10T06:46:38.053 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (8819 ms) 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212268+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi174-80701-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212337+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80239-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212388+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80202-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212440+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi174-79720-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212490+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi174-79750-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212535+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi174-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212617+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi174-79979-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212677+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi174-80206-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212724+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi174-80189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212777+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80463-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212828+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.054 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212883+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi174-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.147 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:46:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:46:38.290 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.212940+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyE api_service: [==========] Running 4 tests from 1 test suite. 2024-02-10T06:46:38.290 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] Global test environment set-up. 2024-02-10T06:46:38.290 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-10T06:46:38.290 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-10T06:46:38.290 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5429 ms) 2024-02-10T06:46:38.290 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.RegisterLate (27 ms) 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: cluster: 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: id: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: health: HEALTH_WARN 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 7 pool(s) do not have an application enabled 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: services: 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mgr: y(active, since 109s), standbys: x 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: data: 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pools: 13 pools, 356 pgs 2024-02-10T06:46:38.291 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: usage: 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pgs: 62.921% pgs unknown 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 224 unknown 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 132 active+clean 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: io: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: client: 639 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: cluster: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: id: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: health: HEALTH_WARN 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 10 pool(s) do not have an application enabled 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: services: 2024-02-10T06:46:38.292 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mgr: y(active, since 111s), standbys: x 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: foo: 10 portals active (1 hosts, 3 zones) 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: data: 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pools: 18 pools, 516 pgs 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: usage: 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pgs: 9.884% pgs unknown 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 59.496% pgs not active 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 307 creating+peering 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 158 active+clean 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 51 unknown 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: io: 2024-02-10T06:46:38.293 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: cluster: 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: id: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: health: HEALTH_WARN 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 18 pool(s) do not have an application enabled 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: services: 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: mgr: y(active, since 113s), standbys: x 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-10T06:46:38.294 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: CPP_smithi174-80178-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.220223+0000 mon.b [INF] from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.220468+0000 mon.b [INF] from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.234628+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.237115+0000 mon.b [INF] from='client.? 172.21.15.174:0/2265060714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.541591+0000 mon.b [INF] from='client.? 172.21.15.174:0/754509825' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.829582+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.830325+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.832501+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.832919+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:36.882901+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.563 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:37.419949+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:38.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:37.420052+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.564 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:37.420132+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.579 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:37.420184+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable"," api_c_read_operations: Running main() from gmock_main.cc 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (297 ms) 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (10 ms) 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (13 ms) 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (32 ms) 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (11 ms) 2024-02-10T06:46:38.580 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (10 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (8 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (4 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-10T06:46:38.581 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (12 ms) 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (12 ms) 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (7 ms) 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (442 ms total) 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (11487 ms total) 2024-02-10T06:46:38.582 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: pgmap v65: 1220 pgs: 743 creating+peering, 319 active+clean, 158 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: osdmap e78: 8 total, 8 up, 8 in 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout:5... 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listed object 50... 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listed object 75... 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listed object 100... 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: listed object 125... 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: saw 150 objects 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_15_[81898]: shutting down. 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.896 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[81899]: starting. 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[81899]: added 25 objects... 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[81899]: added half of the objects 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[81899]: added 50 objects... 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[81899]: added 50 objects 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: process_16_[81899]: shutting down. 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******************************* 2024-02-10T06:46:38.897 INFO:tasks.workunit.client.0.smithi174.stdout: list_parallel: ******* SUCCESS ********** 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: pgmap v65: 1220 pgs: 743 creating+peering, 319 active+clean, 158 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:38.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: osdmap e78: 8 total, 8 up, 8 in 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: pgmap v65: 1220 pgs: 743 creating+peering, 319 active+clean, 158 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi174-80818-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:38.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: osdmap e78: 8 total, 8 up, 8 in 2024-02-10T06:46:38.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:38.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:39.716 INFO:tasks.workunit.client.0.smithi174.stdout:pool": "SimpleWrite_smithi174-79693-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:37.429625+0000 mon.b [INF] from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.001916+0000 mon.b [INF] from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.051797+0000 mon.c [INF] from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.280439+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.052178+0000 client.admin [INF] onexx 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.291519+0000 mon.c [INF] from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.566860+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.567144+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.567439+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.567542+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.567639+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.689995+0000 mon.b [INF] from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:38.890655+0000 mon.c [INF] from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.717 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.174298+0000 mon.a [INF] from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:39.718 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.174465+0000 mon.a [INF] from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:39.718 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.175886+0000 mon.a [INF] from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: onexx 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/250009338' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:39.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: osdmap e79: 8 total, 8 up, 8 in 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: onexx 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/250009338' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: osdmap e79: 8 total, 8 up, 8 in 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:39.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:39.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: onexx 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/250009338' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi174-80178-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi174-80701-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: osdmap e79: 8 total, 8 up, 8 in 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:40.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:40.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:40.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:40.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:40.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:40.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:40.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[29878]: twoxx 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[39523]: twoxx 2024-02-10T06:46:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T06:46:41.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T06:46:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T06:46:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:41 smithi184 ceph-mon[33513]: twoxx 2024-02-10T06:46:41.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: pgmap v68: 700 pgs: 225 creating+peering, 212 active+clean, 263 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1486783183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: pgmap v68: 700 pgs: 225 creating+peering, 212 active+clean, 263 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1486783183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:42.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.9 deep-scrub starts 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.9 deep-scrub ok 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.7 deep-scrub starts 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.7 deep-scrub ok 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.2 deep-scrub starts 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.2 deep-scrub ok 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.3 deep-scrub starts 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.3 deep-scrub ok 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.8 deep-scrub starts 2024-02-10T06:46:42.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.8 deep-scrub ok 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.5 deep-scrub starts 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: 41.5 deep-scrub ok 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: pgmap v70: 668 pgs: 146 creating+peering, 491 active+clean, 31 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 32 MiB/s wr, 466 op/s 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: osdmap e80: 8 total, 8 up, 8 in 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: osdmap e81: 8 total, 8 up, 8 in 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3933439473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.9 deep-scrub starts 2024-02-10T06:46:42.745 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.9 deep-scrub ok 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.7 deep-scrub starts 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.7 deep-scrub ok 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.2 deep-scrub starts 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.2 deep-scrub ok 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.3 deep-scrub starts 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.3 deep-scrub ok 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.8 deep-scrub starts 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.8 deep-scrub ok 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.5 deep-scrub starts 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: 41.5 deep-scrub ok 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: pgmap v70: 668 pgs: 146 creating+peering, 491 active+clean, 31 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 32 MiB/s wr, 466 op/s 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: osdmap e80: 8 total, 8 up, 8 in 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:42.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: osdmap e81: 8 total, 8 up, 8 in 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3933439473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:42.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:42.748 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.749 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: pgmap v68: 700 pgs: 225 creating+peering, 212 active+clean, 263 unknown; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:46:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1486783183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:42.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.9 deep-scrub starts 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.9 deep-scrub ok 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.7 deep-scrub starts 2024-02-10T06:46:42.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.7 deep-scrub ok 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.2 deep-scrub starts 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.2 deep-scrub ok 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.3 deep-scrub starts 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.3 deep-scrub ok 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.8 deep-scrub starts 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.8 deep-scrub ok 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.5 deep-scrub starts 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: 41.5 deep-scrub ok 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: pgmap v70: 668 pgs: 146 creating+peering, 491 active+clean, 31 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 10 MiB/s rd, 32 MiB/s wr, 466 op/s 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: osdmap e80: 8 total, 8 up, 8 in 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: osdmap e81: 8 total, 8 up, 8 in 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3933439473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:42.877 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:42.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:42.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:42.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:42.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:42.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:42.878 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:43.223 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.178988+0000 mon.a [INF] from='client.? cls: Running main() from gmock_main.cc 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [----------] Global test environment set-up. 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [ OK ] NeoRadosCls.DNE (10865 ms) 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (5214 ms) 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [----------] 2 tests from NeoRadosCls (16079 ms total) 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [----------] Global test environment tear-down 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [==========] 2 tests from 1 test suite ran. (16079 ms total) 2024-02-10T06:46:43.224 INFO:tasks.workunit.client.0.smithi174.stdout: cls: [ PASSED ] 2 tests. 2024-02-10T06:46:43.449 INFO:tasks.workunit.client.0.smithi174.stdout: 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.449 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.428654+0000 mon.a [INF] from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:43.449 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.429733+0000 mon.a [INF] from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.430387+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.430982+0000 mon.c [INF] from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.431415+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.432639+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.721674+0000 mon.a [INF] from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.721736+0000 mon.a [INF] from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi174-79720-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.721809+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi174-80160-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.721898+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.728173+0000 mon.c [INF] from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.738519+0000 mon.c [INF] from='client.? 172.21.15.174:0/1486783183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.938838+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.939505+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.939972+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:46:43.450 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.940374+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.940760+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.941183+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.941582+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.942005+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.942387+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:39.942782+0000 mon.b [INF] from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.427017+0000 mon.a [WRN] Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.434415+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.438243+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.438907+0000 mon.a [INF] from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.439117+0000 mon.a [INF] from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.444975+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.456350+0000 mon.b [INF] from='client.? 172.21.15.174:0/3933439473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.456757+0000 mon.b [INF] from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:43.451 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.458234+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: 41.0 deep-scrub starts 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: 41.0 deep-scrub ok 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: 41.4 deep-scrub starts 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: 41.4 deep-scrub ok 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: 41.0 deep-scrub starts 2024-02-10T06:46:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: 41.0 deep-scrub ok 2024-02-10T06:46:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: 41.4 deep-scrub starts 2024-02-10T06:46:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: 41.4 deep-scrub ok 2024-02-10T06:46:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: 41.0 deep-scrub starts 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: 41.0 deep-scrub ok 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: 41.4 deep-scrub starts 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: 41.4 deep-scrub ok 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.458299+0000 mon.b [INF] from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": open_pools_parallel: process_1_[80571]: starting. 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[80571]: creating pool ceph_test_rados_open_pools_parallel.smithi174-80524 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[80571]: created object 0... 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[80571]: created object 25... 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[80571]: created object 49... 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[80571]: finishing. 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_1_[80571]: shutting down. 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[80573]: starting. 2024-02-10T06:46:44.726 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[80573]: rados_pool_create. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[80573]: rados_ioctx_create. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_2_[80573]: shutting down. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: starting. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: creating pool ceph_test_rados_open_pools_parallel.smithi174-80524 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: created object 0... 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: created object 25... 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: created object 49... 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: finishing. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_3_[82136]: shutting down. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[82138]: starting. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[82138]: rados_pool_create. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[82138]: rados_ioctx_create. 2024-02-10T06:46:44.727 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: process_4_[82138]: shutting down. 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******************************* 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: starting. 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: creating pool ceph_test_rados_delete_pools_parallel.smithi174-80605 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: created object 0... 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: created object 25... 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: created object 49... 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: finishing. 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_1_[80707]: shutting down. 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_2_[80709]: starting. 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_2_[80709]: deleting pool ceph_test_rados_delete_pools_parallel.smithi174-80605 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_2_[80709]: shutting down. 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2024-02-10T06:46:44.728 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: starting. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: creating pool ceph_test_rados_delete_pools_parallel.smithi174-80605 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: created object 0... 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: created object 25... 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: created object 49... 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: finishing. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_3_[82139]: shutting down. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[82141]: starting. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[82141]: listing objects. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[82141]: listed object 0... 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[82141]: listed object 25... 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[82141]: saw 50 objects 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_5_[82141]: shutting down. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_4_[82140]: starting. 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_4_[82140]: deleting pool ceph_test_rados_delete_pools_parallel.smithi174-80605 2024-02-10T06:46:44.729 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: process_4_[82140]: shutting down. 2024-02-10T06:46:44.730 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2024-02-10T06:46:44.730 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******************************* 2024-02-10T06:46:44.730 INFO:tasks.workunit.client.0.smithi174.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-10T06:46:45.227 INFO:tasks.workunit.client.0.smithi174.stdout: "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.458689+0000 mon.b [INF] from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.726424+0000 mon.a [INF] from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.726940+0000 mon.a [INF] from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.727922+0000 mon.a [INF] from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.728494+0000 mon.a [INF] from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.730077+0000 mon.a [INF] from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:41.730542+0000 mon.a [INF] from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.205246+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.205445+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.228 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.205594+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.205826+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.206559+0000 mon.b [INF] from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450087+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450269+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450333+0000 mon.a [INF] from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450395+0000 mon.a [INF] from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450462+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450540+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.450612+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.459790+0000 mon.b [INF] from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:42.462179+0000 mon.b [INF] from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.221008+0000 mon.c [INF] from='client.? 172.21.15.174:0/470049480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.229 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.221681+0000 mon.b [INF] from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.222017+0000 mon.b [INF] from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.222301+0000 mon.b [INF] from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.449714+0000 mon.c [INF] from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.449836+0000 client.admin [INF] threexx 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.470428+0000 mon.a [INF] from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.470586+0000 mon.a [INF] from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:45.230 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.471222+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: 41.6 deep-scrub starts 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: 41.6 deep-scrub ok 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: pgmap v73: 876 pgs: 102 creating+peering, 423 active+clean, 351 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 503 op/s 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/470049480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: threexx 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: osdmap e82: 8 total, 8 up, 8 in 2024-02-10T06:46:45.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: 41.6 deep-scrub starts 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: 41.6 deep-scrub ok 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: pgmap v73: 876 pgs: 102 creating+peering, 423 active+clean, 351 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 503 op/s 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/470049480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: threexx 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: osdmap e82: 8 total, 8 up, 8 in 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:45.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: osdmap e83: 8 total, 8 up, 8 in 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:45.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: osdmap e83: 8 total, 8 up, 8 in 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:45.494 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:45.495 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: 41.6 deep-scrub starts 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: 41.6 deep-scrub ok 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi174-79860-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi174-79705-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1426970339' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1041561204' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: pgmap v73: 876 pgs: 102 creating+peering, 423 active+clean, 351 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 503 op/s 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/470049480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: threexx 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: osdmap e82: 8 total, 8 up, 8 in 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:45.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: osdmap e83: 8 total, 8 up, 8 in 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:45.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:45.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:45.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:46.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:46:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:46:46.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T06:46:46.744 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: pgmap v76: 700 pgs: 420 active+clean, 280 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: osdmap e84: 8 total, 8 up, 8 in 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:46.745 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi174-80701-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: fourxx 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi174-80701-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: osdmap e85: 8 total, 8 up, 8 in 2024-02-10T06:46:46.746 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi174-80701-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3229568824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: pgmap v76: 700 pgs: 420 active+clean, 280 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: osdmap e84: 8 total, 8 up, 8 in 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:46.748 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi174-80701-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: fourxx 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi174-80701-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: osdmap e85: 8 total, 8 up, 8 in 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.749 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi174-80701-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:46.750 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:46.750 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3229568824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:46.750 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:46.750 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T06:46:46.750 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.750 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:46.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4032422252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: pgmap v76: 700 pgs: 420 active+clean, 280 unknown; 145 MiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: osdmap e84: 8 total, 8 up, 8 in 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi174-80701-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/408146168' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: fourxx 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi174-79860-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi174-79934-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi174-80178-1"}]': finished 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi174-80818-1"}]': finished 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi174-80701-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: osdmap e85: 8 total, 8 up, 8 in 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi174-80701-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3229568824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T06:46:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:46.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:47.863 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.481825+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi174-79705-3", "field": "max_bytes", "val": "4096"}]: di api_io_pp: Running main() from gmock_main.cc 2024-02-10T06:46:47.863 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-10T06:46:47.863 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-10T06:46:47.863 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: seed 79750 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (289 ms) 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (41 ms) 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (7 ms) 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (5 ms) 2024-02-10T06:46:47.864 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (4 ms) 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (6 ms) 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (4 ms) 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (8 ms) 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (5 ms) 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (6 ms) 2024-02-10T06:46:47.865 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (5 ms) 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (5 ms) 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (11 ms) 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (6 ms) 2024-02-10T06:46:47.866 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (5 ms) 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (4 ms) 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (422 ms total) 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (2593 ms) 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (12 ms) 2024-02-10T06:46:47.867 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-10T06:46:47.868 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-02-10T06:46:47.868 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-10T06:46:47.868 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-02-10T06:46:47.868 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-10T06:46:47.868 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:46:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3229568824' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:47.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: 41.1 deep-scrub starts 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: 41.1 deep-scrub ok 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: osdmap e86: 8 total, 8 up, 8 in 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/743532076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:47.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3229568824' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: 41.1 deep-scrub starts 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: 41.1 deep-scrub ok 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: osdmap e86: 8 total, 8 up, 8 in 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/743532076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:47.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:48.007 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] cmd: Running main() from gmock_main.cc 2024-02-10T06:46:48.007 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-02-10T06:46:48.007 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [----------] Global test environment set-up. 2024-02-10T06:46:48.007 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-02-10T06:46:48.007 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-02-10T06:46:48.007 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (9310 ms) 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (3194 ms) 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (8267 ms) 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [----------] 3 tests from NeoRadosCmd (20771 ms total) 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [----------] Global test environment tear-down 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [==========] 3 tests from 1 test suite ran. (20772 ms total) 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: cmd: [ PASSED ] 3 tests. 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: Running main() from gmock_main.cc 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-02-10T06:46:48.008 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [----------] Global test environment set-up. 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: handle_notify cookie 94859603580720 notify_id 330712481793 notifier_gid 24746 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (10690 ms) 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: Trying... 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: handle_notify cookie 94859616909664 notify_id 352187318272 notifier_gid 25081 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: Waiting for 3.000000000s 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: Timed out. 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: Flushing... 2024-02-10T06:46:48.010 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: Flushed... 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (9998 ms) 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (20688 ms total) 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [----------] Global test environment tear-down 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (20689 ms total) 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: watch_notify: [ PASSED ] 2 tests. 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: Running main() from gmock_main.cc 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] Global test environment set-up. 2024-02-10T06:46:48.011 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (282 ms) 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: no timeout :/ 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (50 ms) 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (10 ms) 2024-02-10T06:46:48.012 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (10 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (7 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.TruncTest (6 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-02-10T06:46:48.013 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.RmXattr (15 ms) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIo.XattrIter (6 ms) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 14 tests from LibRadosIo (404 ms total) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (825 ms) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (20 ms) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (12 ms) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-10T06:46:48.014 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (5 ms) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (8 ms) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] 10 tests from LibRadosIoEC (894 ms total) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [----------] Global test environment tear-down 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [==========] 24 tests from 2 test suites ran. (21198 ms total) 2024-02-10T06:46:48.015 INFO:tasks.workunit.client.0.smithi174.stdout: api_io: [ PASSED ] 24 tests. 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3229568824' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi174-80818-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: 41.1 deep-scrub starts 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: 41.1 deep-scrub ok 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: osdmap e86: 8 total, 8 up, 8 in 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/743532076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:48.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout:spatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.482233+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi174-80160-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.482373+0000 mon.a [INF] from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi174-80189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.482472+0000 mon.a [INF] from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi174-79720-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.482562+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.487958+0000 mon.b [INF] from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.491511+0000 mon.b [INF] from='client.? 172.21.15.174:0/1696292392' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.505106+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi174-79693-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.991217+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.999155+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.473 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.999393+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi174-80818-1"}]: dispatch 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.999621+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:43.999881+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi174-79750-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:44.000187+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi174-80178-1"}]: dispatch 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: got: 2024-02-10T06:46:44.176934+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi174-80701-1"}]': finished 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (10458 ms) 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (21480 ms total) 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-10T06:46:48.474 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (21480 ms total) 2024-02-10T06:46:48.475 INFO:tasks.workunit.client.0.smithi174.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-10T06:46:48.752 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-10T06:46:48.752 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-10T06:46:48.752 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-10T06:46:48.752 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-10T06:46:48.752 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: seed 80189 2024-02-10T06:46:48.752 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (318 ms) 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (7 ms) 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (7 ms) 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (18 ms) 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (351 ms total) 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-10T06:46:48.753 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (852 ms) 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (624 ms) 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (4 ms) 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (14 ms) 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1494 ms total) 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (21756 ms total) 2024-02-10T06:46:48.754 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: Running main() from gmock_main.cc 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] Global test environment set-up. 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.Stat (2588 ms) 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-02-10T06:46:48.762 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (6 ms) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.StatNS (19 ms) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 5 tests from LibRadosStat (2616 ms total) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (831 ms) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (30 ms) 2024-02-10T06:46:48.763 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (864 ms total) 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [----------] Global test environment tear-down 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (21771 ms total) 2024-02-10T06:46:48.764 INFO:tasks.workunit.client.0.smithi174.stdout: api_stat: [ PASSED ] 9 tests. 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: pgmap v79: 828 pgs: 60 creating+peering, 608 active+clean, 160 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi174-80701-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2"}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: osdmap e87: 8 total, 8 up, 8 in 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1632038116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: pgmap v79: 828 pgs: 60 creating+peering, 608 active+clean, 160 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-10T06:46:48.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi174-80701-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2"}]: dispatch 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: osdmap e87: 8 total, 8 up, 8 in 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1632038116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-80178-6", "pg_num": 4}]: dispatch 2024-02-10T06:46:48.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:48.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:48.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:48.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:48.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-80178-6", "pg_num": 4}]: dispatch 2024-02-10T06:46:48.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: pgmap v79: 828 pgs: 60 creating+peering, 608 active+clean, 160 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-02-10T06:46:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi174-80701-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1659128150' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi174-79720-16"}]': finished 2024-02-10T06:46:49.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-4", "overlaypool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3978519392' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2"}]: dispatch 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:49.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: osdmap e87: 8 total, 8 up, 8 in 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1632038116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-80178-6", "pg_num": 4}]: dispatch 2024-02-10T06:46:49.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]': finished 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-80178-6", "pg_num": 4}]': finished 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: osdmap e88: 8 total, 8 up, 8 in 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3425641259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]: dispatch 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: pool 'PoolQuotaPP_smithi174-79705-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T06:46:50.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: osdmap e89: 8 total, 8 up, 8 in 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "overlaypool": "test-rados-api-smithi174-80178-6"}]: dispatch 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:46:50.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]': finished 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-80178-6", "pg_num": 4}]': finished 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: osdmap e88: 8 total, 8 up, 8 in 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3425641259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi174-80818-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi174-80160-7"}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2205935' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi174-80189-7"}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-4-cache", "mode": "writeback"}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi174-79960-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi174-79693-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-80178-6", "pg_num": 4}]': finished 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: osdmap e88: 8 total, 8 up, 8 in 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:46:50.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3425641259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]: dispatch 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: pool 'PoolQuotaPP_smithi174-79705-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T06:46:50.242 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: osdmap e89: 8 total, 8 up, 8 in 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "overlaypool": "test-rados-api-smithi174-80178-6"}]: dispatch 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]: dispatch 2024-02-10T06:46:50.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: pool 'PoolQuotaPP_smithi174-79705-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: osdmap e89: 8 total, 8 up, 8 in 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "overlaypool": "test-rados-api-smithi174-80178-6"}]: dispatch 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:46:50.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: Running main() from gmock_main.cc 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [----------] Global test environment set-up. 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [----------] 3 tests from NeoradosList 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ OK ] NeoradosList.ListObjects (10998 ms) 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-02-10T06:46:50.579 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ OK ] NeoradosList.ListObjectsNS (6681 ms) 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ OK ] NeoradosList.ListObjectsMany (5613 ms) 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: [----------] 3 tests from NeoradosList (23292 ms total) 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: [----------] Global test environment tear-down 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: [==========] 3 tests from 1 test suite ran. (23292 ms total) 2024-02-10T06:46:50.580 INFO:tasks.workunit.client.0.smithi174.stdout: list: [ PASSED ] 3 tests. 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (6 ms) 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-02-10T06:46:51.343 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (6 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (6 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (5 ms) 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-10T06:46:51.344 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2680 ms total) 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24522 ms total) 2024-02-10T06:46:51.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-10T06:46:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: pgmap v82: 752 pgs: 24 creating+peering, 484 active+clean, 244 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-10T06:46:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]: dispatch 2024-02-10T06:46:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:46:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "overlaypool": "test-rados-api-smithi174-80178-6"}]': finished 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]': finished 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1348996227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: osdmap e90: 8 total, 8 up, 8 in 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80178-6", "mode": "writeback"}]: dispatch 2024-02-10T06:46:51.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:51.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:51.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:51.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:51.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:46:51.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: pgmap v82: 752 pgs: 24 creating+peering, 484 active+clean, 244 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-10T06:46:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "overlaypool": "test-rados-api-smithi174-80178-6"}]': finished 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]': finished 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1348996227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: osdmap e90: 8 total, 8 up, 8 in 2024-02-10T06:46:51.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80178-6", "mode": "writeback"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: pgmap v82: 752 pgs: 24 creating+peering, 484 active+clean, 244 unknown; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:51.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "overlaypool": "test-rados-api-smithi174-80178-6"}]': finished 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-4"}]': finished 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1858736799' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2714113603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/891887646' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1348996227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:51.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: osdmap e90: 8 total, 8 up, 8 in 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80178-6", "mode": "writeback"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:46:51.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.369 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: Running main() from gmock_main.cc 2024-02-10T06:46:52.369 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-02-10T06:46:52.369 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [----------] Global test environment set-up. 2024-02-10T06:46:52.369 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-02-10T06:46:52.369 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (16944 ms) 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (8177 ms) 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (25121 ms total) 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [----------] Global test environment tear-down 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (25125 ms total) 2024-02-10T06:46:52.370 INFO:tasks.workunit.client.0.smithi174.stdout: ec_io: [ PASSED ] 2 tests. 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: seed 79934 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-10T06:46:52.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (70 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (1248 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1042 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1145 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (45 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (10 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (6 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (2 ms) 2024-02-10T06:46:52.394 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (3568 ms total) 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (786 ms) 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (1302 ms) 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1087 ms) 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1004 ms) 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (3 ms) 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-10T06:46:52.395 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (3 ms) 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (4192 ms total) 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (25507 ms total) 2024-02-10T06:46:52.396 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: Running main() from gmock_main.cc 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] Global test environment set-up. 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (1346 ms) 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockShared (1247 ms) 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1097 ms) 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1050 ms) 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-10T06:46:52.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (37 ms) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.Unlock (12 ms) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (6 ms) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLock (4798 ms total) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (2085 ms) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (5 ms) 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-10T06:46:52.402 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1062 ms) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (4169 ms total) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [----------] Global test environment tear-down 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (25521 ms total) 2024-02-10T06:46:52.403 INFO:tasks.workunit.client.0.smithi174.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: pgmap v85: 688 pgs: 96 creating+peering, 592 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 3 op/s 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80178-6", "mode": "writeback"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]': finished 2024-02-10T06:46:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: osdmap e91: 8 total, 8 up, 8 in 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3013081117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:52 smithi184 ceph-mon[33513]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: pgmap v85: 688 pgs: 96 creating+peering, 592 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 3 op/s 2024-02-10T06:46:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80178-6", "mode": "writeback"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: osdmap e91: 8 total, 8 up, 8 in 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3013081117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:52.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[29878]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: pgmap v85: 688 pgs: 96 creating+peering, 592 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 3 op/s 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi174-80206-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80178-6", "mode": "writeback"}]': finished 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-4", "tierpool": "test-rados-api-smithi174-79953-4-cache"}]': finished 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:52.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi174-79750-22"}]': finished 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi174-79960-1","app":"app1","key":"key1"}]': finished 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: osdmap e91: 8 total, 8 up, 8 in 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3013081117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi174-80701-2"}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:52.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:52 smithi174 ceph-mon[39523]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:53.419 INFO:tasks.workunit.client.0.smithi174.stdout: pool: Running main() from gmock_main.cc 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [----------] Global test environment set-up. 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ OK ] NeoRadosPools.PoolList (9255 ms) 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-02-10T06:46:53.420 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (4914 ms) 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (3514 ms) 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4607 ms) 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1772 ms) 2024-02-10T06:46:53.421 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-02-10T06:46:53.422 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2055 ms) 2024-02-10T06:46:53.422 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [----------] 6 tests from NeoRadosPools (26118 ms total) 2024-02-10T06:46:53.422 INFO:tasks.workunit.client.0.smithi174.stdout: pool: 2024-02-10T06:46:53.422 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [----------] Global test environment tear-down 2024-02-10T06:46:53.422 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [==========] 6 tests from 1 test suite ran. (26118 ms total) 2024-02-10T06:46:53.423 INFO:tasks.workunit.client.0.smithi174.stdout: pool: [ PASSED ] 6 tests. 2024-02-10T06:46:53.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: osdmap e92: 8 total, 8 up, 8 in 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2751521346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: osdmap e92: 8 total, 8 up, 8 in 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2751521346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:53.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:53.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:53.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/273715983' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi174-79860-10"}]': finished 2024-02-10T06:46:53.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3959542356' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi174-79934-10"}]': finished 2024-02-10T06:46:53.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3618255880' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi174-80701-2"}]': finished 2024-02-10T06:46:53.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:53.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi174-79693-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:53.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: osdmap e92: 8 total, 8 up, 8 in 2024-02-10T06:46:53.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4113398796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:53.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]: dispatch 2024-02-10T06:46:53.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2751521346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:53.743 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: pgmap v88: 720 pgs: 200 unknown, 64 creating+peering, 456 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T06:46:54.883 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:54.883 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: osdmap e93: 8 total, 8 up, 8 in 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3719111669' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.884 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: pgmap v88: 720 pgs: 200 unknown, 64 creating+peering, 456 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: osdmap e93: 8 total, 8 up, 8 in 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3719111669' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: pgmap v88: 720 pgs: 200 unknown, 64 creating+peering, 456 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi174-80818-2"}]': finished 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: osdmap e93: 8 total, 8 up, 8 in 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3719111669' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:54.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (2937 ms) 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (2941 ms total) 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: 2024-02-10T06:46:55.081 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (15 ms) 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3978 ms) 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (5 ms) 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727096864 notify_id 390842023939 notifier_gid 15249 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727099040 notify_id 390842023941 notifier_gid 15249 2024-02-10T06:46:55.082 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (11 ms) 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727099040 notify_id 390842023938 notifier_gid 15249 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (8 ms) 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727099040 notify_id 390842023942 notifier_gid 15249 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (14 ms) 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727099040 notify_id 390842023940 notifier_gid 15249 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (11 ms) 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727099040 notify_id 390842023939 notifier_gid 15249 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (16 ms) 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-10T06:46:55.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: trying... 2024-02-10T06:46:55.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727099040 notify_id 390842023941 notifier_gid 15249 2024-02-10T06:46:55.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: timed out 2024-02-10T06:46:55.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2024-02-10T06:46:56.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:46:56.778 INFO:tasks.workunit.client.0.smithi174.stdout: api_w api_watch_notify: Running main() from gmock_main.cc 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify_test_cb 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (284 ms) 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-10T06:46:56.779 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94692839855008 err -107 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1067 ms) 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94692839866416 err -107 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (31 ms) 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 335007449090 cookie 94692839884096 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (6 ms) 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 335007449088 cookie 94692839894432 2024-02-10T06:46:56.780 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (5 ms) 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 335007449091 cookie 94692839894432 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (5 ms) 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 335007449092 cookie 94692839891040 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 335007449092 cookie 94692839894432 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (6 ms) 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 335007449088 cookie 94692839892368 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 339302416385 cookie 94692839892368 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94692839925440 err -107 2024-02-10T06:46:56.781 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_cb from 24682 notify_id 365072220162 cookie 94692839925440 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94692839910560 err -107 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (5 ms) 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9421 ms total) 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: watch_notify_test_cb 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1178 ms) 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1178 ms total) 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-10T06:46:56.782 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (29777 ms total) 2024-02-10T06:46:56.783 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: osdmap e94: 8 total, 8 up, 8 in 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4057406586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/269204698' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: osdmap e94: 8 total, 8 up, 8 in 2024-02-10T06:46:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4057406586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/269204698' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi174-79979-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: osdmap e94: 8 total, 8 up, 8 in 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4057406586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4250173234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/269204698' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:57.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:57.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]: dispatch 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: pgmap v91: 520 pgs: 160 unknown, 360 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: osdmap e95: 8 total, 8 up, 8 in 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4180576231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: osdmap e96: 8 total, 8 up, 8 in 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3065864479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: pgmap v91: 520 pgs: 160 unknown, 360 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: osdmap e95: 8 total, 8 up, 8 in 2024-02-10T06:46:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4180576231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: osdmap e96: 8 total, 8 up, 8 in 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3065864479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.243 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:46:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: pgmap v91: 520 pgs: 160 unknown, 360 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi174-79693-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi174-80206-12"}]': finished 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: osdmap e95: 8 total, 8 up, 8 in 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4180576231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi174-80818-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi174-79986-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: osdmap e96: 8 total, 8 up, 8 in 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3065864479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:46:58.345 INFO:tasks.workunit.client.0.smithi174.stdout:api_service: 2024-02-10T06:46:58.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: data: 2024-02-10T06:46:58.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pools: 40 pools, 1220 pgs 2024-02-10T06:46:58.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: objects: 200 objects, 582 KiB 2024-02-10T06:46:58.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: usage: 239 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:46:58.345 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: pgs: 12.951% pgs unknown 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 60.902% pgs not active 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 743 creating+peering 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 319 active+clean 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 158 unknown 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: io: 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: client: 853 B/s rd, 0 B/s wr, 1 op/s rd, 0 op/s wr 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.StatusFormat (5811 ms) 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ OK ] LibRadosService.Status (20022 ms) 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] 4 tests from LibRadosService (31290 ms total) 2024-02-10T06:46:58.346 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: 2024-02-10T06:46:58.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [----------] Global test environment tear-down 2024-02-10T06:46:58.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31290 ms total) 2024-02-10T06:46:58.347 INFO:tasks.workunit.client.0.smithi174.stdout: api_service: [ PASSED ] 4 tests. 2024-02-10T06:46:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: pgmap v94: 712 pgs: 96 unknown, 167 creating+peering, 449 active+clean; 145 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2 op/s 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[29878]: osdmap e97: 8 total, 8 up, 8 in 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: pgmap v94: 712 pgs: 96 unknown, 167 creating+peering, 449 active+clean; 145 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2 op/s 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T06:46:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:59.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:46:58 smithi174 ceph-mon[39523]: osdmap e97: 8 total, 8 up, 8 in 2024-02-10T06:46:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: pgmap v94: 712 pgs: 96 unknown, 167 creating+peering, 449 active+clean; 145 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2 op/s 2024-02-10T06:46:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:46:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T06:46:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:46:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:46:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:46:58 smithi184 ceph-mon[33513]: osdmap e97: 8 total, 8 up, 8 in 2024-02-10T06:46:59.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-10T06:46:59.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-10T06:46:59.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-10T06:46:59.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-10T06:46:59.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5046 ms) 2024-02-10T06:46:59.386 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (37 ms) 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (21802 ms) 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: attempt 0 of 20 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: failed to deregister: 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: { 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "epoch": 19, 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "modified": "2024-02-10T06:46:54.915733+0000", 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "services": { 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "laundry": { 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "daemons": { 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "summary": "", 2024-02-10T06:46:59.387 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "close-test-pid80335": { 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "start_epoch": 19, 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "start_stamp": "2024-02-10T06:46:54.006213+0000", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "gid": 27041, 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "addr": "172.21.15.174:0/1252201802", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "metadata": { 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "arch": "x86_64", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_release": "squid", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_version_short": "19.0.0-1361-g290212a1", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro": "centos", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro_version": "9", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "foo": "bar", 2024-02-10T06:46:59.388 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "hostname": "smithi174", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "kernel_version": "5.14.0-419.el9.x86_64", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "mem_total_kb": "32488372", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "os": "Linux", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "this": "that" 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: }, 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "task_status": {} 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: }, 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "pid80232": { 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "start_epoch": 18, 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "start_stamp": "2024-02-10T06:46:38.336512+0000", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "gid": 25120, 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "addr": "172.21.15.174:0/821664939", 2024-02-10T06:46:59.389 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "metadata": { 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "arch": "x86_64", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_release": "squid", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_version_short": "19.0.0-1361-g290212a1", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro": "centos", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro_version": "9", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "foo": "bar", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "hostname": "smithi174", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "kernel_version": "5.14.0-419.el9.x86_64", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "mem_total_kb": "32488372", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "os": "Linux", 2024-02-10T06:46:59.390 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "this": "that" 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: }, 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "task_status": {} 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: }, 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "rgw": { 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "daemons": { 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "summary": "", 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "24329": { 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "start_epoch": 12, 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "start_stamp": "2024-02-10T06:44:10.302646+0000", 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "gid": 24329, 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "addr": "172.21.15.174:0/2995390865", 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "metadata": { 2024-02-10T06:46:59.391 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "arch": "x86_64", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_release": "squid", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "ceph_version_short": "19.0.0-1361-g290212a1", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "container_hostname": "smithi174", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro": "centos", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "distro_version": "8", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "hostname": "smithi174", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "id": "foo.a", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024", 2024-02-10T06:46:59.392 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "kernel_version": "5.14.0-419.el9.x86_64", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "mem_total_kb": "32488372", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "num_handles": "1", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "os": "Linux", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "pid": "7", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "realm_id": "", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "realm_name": "", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "zone_id": "10ad3cb8-cd7f-4674-b2ff-2b18aa48203f", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "zone_name": "default", 2024-02-10T06:46:59.393 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "zonegroup_id": "11a1951c-3cdb-47d2-a8bc-6629ac2842b5", 2024-02-10T06:47:00.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:00 smithi184 ceph-mon[33513]: osdmap e98: 8 total, 8 up, 8 in 2024-02-10T06:47:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3476904638' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[29878]: osdmap e98: 8 total, 8 up, 8 in 2024-02-10T06:47:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3476904638' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi174-80818-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[39523]: osdmap e98: 8 total, 8 up, 8 in 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3476904638' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: pgmap v97: 528 pgs: 136 unknown, 58 creating+peering, 334 active+clean; 145 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T06:47:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4125576904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[29878]: osdmap e99: 8 total, 8 up, 8 in 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: pgmap v97: 528 pgs: 136 unknown, 58 creating+peering, 334 active+clean; 145 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4125576904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:01 smithi174 ceph-mon[39523]: osdmap e99: 8 total, 8 up, 8 in 2024-02-10T06:47:01.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: pgmap v97: 528 pgs: 136 unknown, 58 creating+peering, 334 active+clean; 145 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T06:47:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4125576904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:01 smithi184 ceph-mon[33513]: osdmap e99: 8 total, 8 up, 8 in 2024-02-10T06:47:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: pgmap v99: 560 pgs: 134 creating+peering, 426 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:47:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2978502944' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: osdmap e100: 8 total, 8 up, 8 in 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: pgmap v99: 560 pgs: 134 creating+peering, 426 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2978502944' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: osdmap e100: 8 total, 8 up, 8 in 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:02.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: pgmap v99: 560 pgs: 134 creating+peering, 426 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2978502944' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: osdmap e100: 8 total, 8 up, 8 in 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]': finished 2024-02-10T06:47:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: osdmap e101: 8 total, 8 up, 8 in 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/214091258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]': finished 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: osdmap e101: 8 total, 8 up, 8 in 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/214091258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:03.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:03.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi174-79693-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-7"}]': finished 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: osdmap e101: 8 total, 8 up, 8 in 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/214091258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:04.501 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-10T06:47:04.501 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: }, 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: "task_status": {} 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: } 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: attempt 1 of 20 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10539 ms) 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (37424 ms total) 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (37425 ms total) 2024-02-10T06:47:04.502 INFO:tasks.workunit.client.0.smithi174.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: pgmap v102: 624 pgs: 128 unknown, 81 creating+peering, 415 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: osdmap e102: 8 total, 8 up, 8 in 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:04.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2123571767' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4"}]: dispatch 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: pgmap v102: 624 pgs: 128 unknown, 81 creating+peering, 415 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: osdmap e102: 8 total, 8 up, 8 in 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:04.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2123571767' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:04.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4"}]: dispatch 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: pgmap v102: 624 pgs: 128 unknown, 81 creating+peering, 415 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: osdmap e102: 8 total, 8 up, 8 in 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]: dispatch 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2123571767' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:05.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4"}]: dispatch 2024-02-10T06:47:06.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2936518189' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]': finished 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2123571767' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4"}]': finished 2024-02-10T06:47:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: osdmap e103: 8 total, 8 up, 8 in 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1609354984' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6"}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1472122820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6"}]': finished 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: osdmap e104: 8 total, 8 up, 8 in 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi174-80178-6", "pool2": "test-rados-api-smithi174-80178-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/725596393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2936518189' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]': finished 2024-02-10T06:47:06.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2123571767' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4"}]': finished 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: osdmap e103: 8 total, 8 up, 8 in 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1609354984' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6"}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1472122820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6"}]': finished 2024-02-10T06:47:06.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: osdmap e104: 8 total, 8 up, 8 in 2024-02-10T06:47:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi174-80178-6", "pool2": "test-rados-api-smithi174-80178-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.744 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/725596393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2936518189' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:06.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T06:47:06.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:06.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-7"}]': finished 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2123571767' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4"}]': finished 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: osdmap e103: 8 total, 8 up, 8 in 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1609354984' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6"}]: dispatch 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1472122820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:06.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi174-80178-4", "tierpool": "test-rados-api-smithi174-80178-6"}]': finished 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi174-79693-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi174-79705-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: osdmap e104: 8 total, 8 up, 8 in 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi174-80178-6", "pool2": "test-rados-api-smithi174-80178-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T06:47:06.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/725596393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: pgmap v105: 688 pgs: 256 unknown, 17 creating+peering, 415 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi174-80178-6", "pool2": "test-rados-api-smithi174-80178-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/725596393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: osdmap e105: 8 total, 8 up, 8 in 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: pgmap v105: 688 pgs: 256 unknown, 17 creating+peering, 415 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi174-80178-6", "pool2": "test-rados-api-smithi174-80178-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/725596393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: osdmap e105: 8 total, 8 up, 8 in 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: pgmap v105: 688 pgs: 256 unknown, 17 creating+peering, 415 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2972910556' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi174-80178-6", "pool2": "test-rados-api-smithi174-80178-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/725596393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80476-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3572195200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: osdmap e105: 8 total, 8 up, 8 in 2024-02-10T06:47:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]: dispatch 2024-02-10T06:47:07.895 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: Running main() from gmock_main.cc 2024-02-10T06:47:07.895 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-02-10T06:47:07.895 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [----------] Global test environment set-up. 2024-02-10T06:47:07.895 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-02-10T06:47:07.895 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-02-10T06:47:07.895 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (17993 ms) 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (8140 ms) 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (14502 ms) 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [----------] 3 tests from NeoradosECList (40635 ms total) 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [----------] Global test environment tear-down 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (40635 ms total) 2024-02-10T06:47:07.896 INFO:tasks.workunit.client.0.smithi174.stdout: ec_list: [ PASSED ] 3 tests. 2024-02-10T06:47:07.956 INFO:tasks.workunit.client.0.smithi174.stdout:atch_notify_pp: flushed 2024-02-10T06:47:07.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3005 ms) 2024-02-10T06:47:07.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-10T06:47:07.956 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: trying... 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727113616 notify_id 403726925825 notifier_gid 15249 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: timed out 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushed 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3032 ms) 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: List watches 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727126592 notify_id 416611827714 notifier_gid 15249 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 done 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: watch_check 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: unwatch2 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: done 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3014 ms) 2024-02-10T06:47:07.957 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: List watches 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: handle_notify cookie 94594727126592 notify_id 425201762307 notifier_gid 15249 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: notify2 done 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: watch_check 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: unwatch2 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: flushing 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: done 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3021 ms) 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16142 ms total) 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (40921 ms total) 2024-02-10T06:47:07.958 INFO:tasks.workunit.client.0.smithi174.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-10T06:47:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:47:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:47:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: pgmap v108: 676 pgs: 32 creating+peering, 96 unknown, 548 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: osdmap e106: 8 total, 8 up, 8 in 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3548978607' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:09.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: pgmap v108: 676 pgs: 32 creating+peering, 96 unknown, 548 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:47:09.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:09.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:09.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: osdmap e106: 8 total, 8 up, 8 in 2024-02-10T06:47:09.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:09.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3548978607' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: pgmap v108: 676 pgs: 32 creating+peering, 96 unknown, 548 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi174-80818-3"}]': finished 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: osdmap e106: 8 total, 8 up, 8 in 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3548978607' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:09.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:09.743 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: Running main() from gmock_main.cc 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [----------] Global test environment set-up. 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (10697 ms) 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (6717 ms) 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-02-10T06:47:10.802 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (5855 ms) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (2809 ms) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (4824 ms) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3978 ms) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (8597 ms) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (43477 ms total) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [----------] Global test environment tear-down 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (43477 ms total) 2024-02-10T06:47:10.803 INFO:tasks.workunit.client.0.smithi174.stdout: write_operations: [ PASSED ] 7 tests. 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/798388238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2461438150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: osdmap e107: 8 total, 8 up, 8 in 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: osdmap e108: 8 total, 8 up, 8 in 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/798388238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2461438150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: osdmap e107: 8 total, 8 up, 8 in 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: osdmap e108: 8 total, 8 up, 8 in 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi174-79705-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/798388238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2461438150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: osdmap e107: 8 total, 8 up, 8 in 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:10.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: osdmap e108: 8 total, 8 up, 8 in 2024-02-10T06:47:10.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:10.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: pgmap v110: 548 pgs: 32 creating+peering, 128 unknown, 388 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:47:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:47:11.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.7 deep-scrub starts 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.7 deep-scrub ok 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.2 deep-scrub starts 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.2 deep-scrub ok 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.5 deep-scrub starts 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.3 deep-scrub starts 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.5 deep-scrub ok 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: 41.3 deep-scrub ok 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]': finished 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: osdmap e109: 8 total, 8 up, 8 in 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]: dispatch 2024-02-10T06:47:11.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]: dispatch 2024-02-10T06:47:11.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: pgmap v110: 548 pgs: 32 creating+peering, 128 unknown, 388 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:47:11.992 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.7 deep-scrub starts 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.7 deep-scrub ok 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.2 deep-scrub starts 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.2 deep-scrub ok 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.5 deep-scrub starts 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.3 deep-scrub starts 2024-02-10T06:47:11.993 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.5 deep-scrub ok 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: 41.3 deep-scrub ok 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]': finished 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: osdmap e109: 8 total, 8 up, 8 in 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]: dispatch 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]: dispatch 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: pgmap v110: 548 pgs: 32 creating+peering, 128 unknown, 388 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:11.994 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.0"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.1"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.2"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.3"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.4"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.5"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.6"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:47:11.995 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.7"}]: dispatch 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.8"}]: dispatch 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "41.9"}]: dispatch 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.7 deep-scrub starts 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.7 deep-scrub ok 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.2 deep-scrub starts 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.2 deep-scrub ok 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.5 deep-scrub starts 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.3 deep-scrub starts 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.5 deep-scrub ok 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: 41.3 deep-scrub ok 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-9"}]': finished 2024-02-10T06:47:11.996 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi174-79693-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:11.997 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:11.997 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: osdmap e109: 8 total, 8 up, 8 in 2024-02-10T06:47:11.997 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]: dispatch 2024-02-10T06:47:11.997 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]: dispatch 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-10T06:47:12.818 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (10888 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6717 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4020 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (4630 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (6695 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4634 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (8112 ms) 2024-02-10T06:47:12.819 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (45698 ms total) 2024-02-10T06:47:12.820 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: 2024-02-10T06:47:12.820 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-10T06:47:12.820 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (45698 ms total) 2024-02-10T06:47:12.820 INFO:tasks.workunit.client.0.smithi174.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.8 deep-scrub starts 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.8 deep-scrub ok 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: pgmap v114: 612 pgs: 1 active+clean+snaptrim, 36 creating+activating, 53 creating+peering, 60 unknown, 462 active+clean; 193 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 12 MiB/s wr, 1 op/s 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.9 deep-scrub starts 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.9 deep-scrub ok 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.0 deep-scrub starts 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.0 deep-scrub ok 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.4 deep-scrub starts 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: 41.4 deep-scrub ok 2024-02-10T06:47:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]': finished 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: osdmap e110: 8 total, 8 up, 8 in 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/448142873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:12.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.8 deep-scrub starts 2024-02-10T06:47:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.8 deep-scrub ok 2024-02-10T06:47:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: pgmap v114: 612 pgs: 1 active+clean+snaptrim, 36 creating+activating, 53 creating+peering, 60 unknown, 462 active+clean; 193 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 12 MiB/s wr, 1 op/s 2024-02-10T06:47:12.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.9 deep-scrub starts 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.9 deep-scrub ok 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.0 deep-scrub starts 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.0 deep-scrub ok 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.4 deep-scrub starts 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: 41.4 deep-scrub ok 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]': finished 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: osdmap e110: 8 total, 8 up, 8 in 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/448142873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:12.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.8 deep-scrub starts 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.8 deep-scrub ok 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: pgmap v114: 612 pgs: 1 active+clean+snaptrim, 36 creating+activating, 53 creating+peering, 60 unknown, 462 active+clean; 193 MiB data, 836 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 12 MiB/s wr, 1 op/s 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.9 deep-scrub starts 2024-02-10T06:47:12.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.9 deep-scrub ok 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.0 deep-scrub starts 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.0 deep-scrub ok 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.4 deep-scrub starts 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: 41.4 deep-scrub ok 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-9", "mode": "writeback"}]': finished 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: osdmap e110: 8 total, 8 up, 8 in 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/448142873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:12.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:12.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:12.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: 41.1 deep-scrub starts 2024-02-10T06:47:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: 41.1 deep-scrub ok 2024-02-10T06:47:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: 41.6 deep-scrub starts 2024-02-10T06:47:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: 41.6 deep-scrub ok 2024-02-10T06:47:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:13.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: osdmap e111: 8 total, 8 up, 8 in 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:13.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/677798948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:13.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:13.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: 41.1 deep-scrub starts 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: 41.1 deep-scrub ok 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: 41.6 deep-scrub starts 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: 41.6 deep-scrub ok 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:13.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: osdmap e111: 8 total, 8 up, 8 in 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/677798948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: 41.1 deep-scrub starts 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: 41.1 deep-scrub ok 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: 41.6 deep-scrub starts 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: 41.6 deep-scrub ok 2024-02-10T06:47:13.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi174-79705-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: osdmap e111: 8 total, 8 up, 8 in 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/677798948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:13.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]: dispatch 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] Global test environment set-up. 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-10T06:47:14.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2024-02-10T06:46:27.497+0000 7f5934ab8780 -1 failed for service _ceph-mon._tcp 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2024-02-10T06:46:27.497+0000 7f5934ab8780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (576 ms) 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2009 ms) 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2585 ms total) 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x5640e9965720 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 0x5640e9c90e50 2024-02-10T06:47:14.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: started 2 aios 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x5640e9965720 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: waiting 0x5640e9c90e50 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: done. 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (10226 ms) 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (10226 ms total) 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.Exec (135 ms) 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (17 ms) 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-10T06:47:14.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (6 ms) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.Applications (6975 ms) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (21548 ms) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] 8 tests from LibRadosMisc (28681 ms total) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [----------] Global test environment tear-down 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (47119 ms total) 2024-02-10T06:47:14.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-10T06:47:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:14 smithi184 ceph-mon[33513]: pgmap v117: 516 pgs: 24 creating+activating, 96 unknown, 396 active+clean; 217 MiB data, 886 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 1 op/s 2024-02-10T06:47:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:14 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]': finished 2024-02-10T06:47:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:14 smithi184 ceph-mon[33513]: osdmap e112: 8 total, 8 up, 8 in 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[29878]: pgmap v117: 516 pgs: 24 creating+activating, 96 unknown, 396 active+clean; 217 MiB data, 886 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 1 op/s 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]': finished 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[29878]: osdmap e112: 8 total, 8 up, 8 in 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[39523]: pgmap v117: 516 pgs: 24 creating+activating, 96 unknown, 396 active+clean; 217 MiB data, 886 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 1 op/s 2024-02-10T06:47:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:15.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:15.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi174-79693-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:15.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-9"}]': finished 2024-02-10T06:47:15.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:14 smithi174 ceph-mon[39523]: osdmap e112: 8 total, 8 up, 8 in 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: osdmap e113: 8 total, 8 up, 8 in 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1110372779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi174-79705-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.033 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.034 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: osdmap e113: 8 total, 8 up, 8 in 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1110372779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi174-79705-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi174-79979-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: osdmap e113: 8 total, 8 up, 8 in 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1110372779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi174-79705-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:16.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: pgmap v120: 460 pgs: 168 unknown, 292 active+clean; 217 MiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1110372779' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi174-79705-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: osdmap e114: 8 total, 8 up, 8 in 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2936624769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi174-79693-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:17.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: pgmap v120: 460 pgs: 168 unknown, 292 active+clean; 217 MiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1110372779' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi174-79705-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: osdmap e114: 8 total, 8 up, 8 in 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2936624769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi174-79693-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: pgmap v120: 460 pgs: 168 unknown, 292 active+clean; 217 MiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1110372779' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi174-79705-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-80017-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: osdmap e114: 8 total, 8 up, 8 in 2024-02-10T06:47:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2936624769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi174-79693-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:18.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:18.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2936624769' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi174-79693-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:18.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:18.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: osdmap e115: 8 total, 8 up, 8 in 2024-02-10T06:47:18.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T06:47:18.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T06:47:18.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:47:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2936624769' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi174-79693-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: osdmap e115: 8 total, 8 up, 8 in 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T06:47:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2936624769' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi174-79693-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: osdmap e115: 8 total, 8 up, 8 in 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T06:47:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: pgmap v123: 524 pgs: 64 creating+peering, 96 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 3.2 KiB/s wr, 5 op/s 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]': finished 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: osdmap e116: 8 total, 8 up, 8 in 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3715184200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:19.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: pgmap v123: 524 pgs: 64 creating+peering, 96 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 3.2 KiB/s wr, 5 op/s 2024-02-10T06:47:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]': finished 2024-02-10T06:47:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: osdmap e116: 8 total, 8 up, 8 in 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3715184200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: pgmap v123: 524 pgs: 64 creating+peering, 96 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 3.2 KiB/s wr, 5 op/s 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache", "force_nonempty":""}]': finished 2024-02-10T06:47:19.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: osdmap e116: 8 total, 8 up, 8 in 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3715184200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:19.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:19.799 INFO:tasks.workunit.client.0.smithi174.stdout: misc: Running main() from gmock_main.cc 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [----------] Global test environment set-up. 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.Version (10732 ms) 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (5218 ms) 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.LongName (3510 ms) 2024-02-10T06:47:19.800 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3843 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2791 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (5576 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.Exec (4481 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (4449 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3909 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3011 ms) 2024-02-10T06:47:19.801 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2036 ms) 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2962 ms) 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [----------] 12 tests from NeoRadosMisc (52519 ms total) 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [----------] Global test environment tear-down 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [==========] 12 tests from 1 test suite ran. (52519 ms total) 2024-02-10T06:47:19.802 INFO:tasks.workunit.client.0.smithi174.stdout: misc: [ PASSED ] 12 tests. 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: osdmap e117: 8 total, 8 up, 8 in 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]: dispatch 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2330714866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi174-79693-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]: dispatch 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]': finished 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2330714866' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi174-79693-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]': finished 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: osdmap e118: 8 total, 8 up, 8 in 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]: dispatch 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]: dispatch 2024-02-10T06:47:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: osdmap e117: 8 total, 8 up, 8 in 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]: dispatch 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2330714866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi174-79693-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]: dispatch 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]': finished 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2330714866' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi174-79693-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]': finished 2024-02-10T06:47:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: osdmap e118: 8 total, 8 up, 8 in 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-80017-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: osdmap e117: 8 total, 8 up, 8 in 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1651944314' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2330714866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi174-79693-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]: dispatch 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-11"}]': finished 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2330714866' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi174-79693-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-80017-10", "tierpool":"test-rados-api-smithi174-80017-10-cache"}]': finished 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: osdmap e118: 8 total, 8 up, 8 in 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]: dispatch 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:20.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]: dispatch 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: pgmap v126: 524 pgs: 32 creating+peering, 128 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]': finished 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: osdmap e119: 8 total, 8 up, 8 in 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2057854081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: pgmap v126: 524 pgs: 32 creating+peering, 128 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]': finished 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: osdmap e119: 8 total, 8 up, 8 in 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2057854081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: pgmap v126: 524 pgs: 32 creating+peering, 128 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-11", "mode": "writeback"}]': finished 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: osdmap e119: 8 total, 8 up, 8 in 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2057854081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:21.803 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: Running main() from gmock_main.cc 2024-02-10T06:47:21.803 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] Global test environment set-up. 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11643 ms) 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6145 ms) 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3288 ms) 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2571 ms) 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (4010 ms) 2024-02-10T06:47:21.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (7602 ms) 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5709 ms) 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (6137 ms) 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7762 ms) 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] 9 tests from LibRadosPools (54867 ms total) 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [----------] Global test environment tear-down 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (54867 ms total) 2024-02-10T06:47:21.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-10T06:47:23.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: pgmap v129: 492 pgs: 59 creating+peering, 25 unknown, 408 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 257 KiB/s wr, 4 op/s 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: osdmap e120: 8 total, 8 up, 8 in 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1813533203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:23.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: pgmap v129: 492 pgs: 59 creating+peering, 25 unknown, 408 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 257 KiB/s wr, 4 op/s 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: osdmap e120: 8 total, 8 up, 8 in 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1813533203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: pgmap v129: 492 pgs: 59 creating+peering, 25 unknown, 408 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 257 KiB/s wr, 4 op/s 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi174-79705-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: osdmap e120: 8 total, 8 up, 8 in 2024-02-10T06:47:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1813533203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: osdmap e121: 8 total, 8 up, 8 in 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:24.075 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: osdmap e121: 8 total, 8 up, 8 in 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:24.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:23 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi174-79693-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: osdmap e121: 8 total, 8 up, 8 in 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]: dispatch 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:24.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:23 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:47:25.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: pgmap v132: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 6 op/s 2024-02-10T06:47:25.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:25.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]': finished 2024-02-10T06:47:25.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: osdmap e122: 8 total, 8 up, 8 in 2024-02-10T06:47:25.562 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/198151586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: osdmap e123: 8 total, 8 up, 8 in 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:25.563 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: pgmap v132: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 6 op/s 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]': finished 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: osdmap e122: 8 total, 8 up, 8 in 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/198151586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: osdmap e123: 8 total, 8 up, 8 in 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:25.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: pgmap v132: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 6 op/s 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-11"}]': finished 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: osdmap e122: 8 total, 8 up, 8 in 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/198151586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi174-79705-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: osdmap e123: 8 total, 8 up, 8 in 2024-02-10T06:47:25.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:25.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.308 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:47:26.507 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/150188848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.507 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: pgmap v135: 388 pgs: 32 creating+peering, 64 unknown, 292 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:26.507 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: osdmap e124: 8 total, 8 up, 8 in 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/150188848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: pgmap v135: 388 pgs: 32 creating+peering, 64 unknown, 292 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: osdmap e124: 8 total, 8 up, 8 in 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:26.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/150188848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: pgmap v135: 388 pgs: 32 creating+peering, 64 unknown, 292 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi174-79693-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3593255788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: osdmap e124: 8 total, 8 up, 8 in 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:27.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:27.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:27.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:27.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: osdmap e125: 8 total, 8 up, 8 in 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1559986890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:27.807 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: osdmap e125: 8 total, 8 up, 8 in 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1559986890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:27.808 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:27.809 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: Running main() from gmock_main.cc 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [----------] Global test environment set-up. 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (10700 ms) 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-02-10T06:47:27.839 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (6718 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (4026 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (4618 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (4841 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3978 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3513 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3647 ms) 2024-02-10T06:47:27.840 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3432 ms) 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3026 ms) 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3039 ms) 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2957 ms) 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3006 ms) 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3025 ms) 2024-02-10T06:47:27.841 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (60527 ms total) 2024-02-10T06:47:27.842 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: 2024-02-10T06:47:27.842 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [----------] Global test environment tear-down 2024-02-10T06:47:27.842 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (60527 ms total) 2024-02-10T06:47:27.842 INFO:tasks.workunit.client.0.smithi174.stdout: read_operations: [ PASSED ] 14 tests. 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: Running main() from gmock_main.cc 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: [----------] Global test environment set-up. 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: [----------] 14 tests from NeoRadosIo 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.Limits (10854 ms) 2024-02-10T06:47:27.846 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (6952 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.ReadOp (3790 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.SparseRead (2573 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3254 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (5507 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (5614 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3659 ms) 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-02-10T06:47:27.847 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3434 ms) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.Trunc (3029 ms) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.Remove (3020 ms) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2958 ms) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.RmXattr (2996 ms) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3042 ms) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [----------] 14 tests from NeoRadosIo (60682 ms total) 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: 2024-02-10T06:47:27.848 INFO:tasks.workunit.client.0.smithi174.stdout: io: [----------] Global test environment tear-down 2024-02-10T06:47:27.849 INFO:tasks.workunit.client.0.smithi174.stdout: io: [==========] 14 tests from 1 test suite ran. (60682 ms total) 2024-02-10T06:47:27.849 INFO:tasks.workunit.client.0.smithi174.stdout: io: [ PASSED ] 14 tests. 2024-02-10T06:47:27.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi174-79979-10"}]': finished 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: osdmap e125: 8 total, 8 up, 8 in 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1559986890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:27.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:28.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:47:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:47:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: pgmap v138: 452 pgs: 96 creating+peering, 32 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s wr, 3 op/s 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: osdmap e126: 8 total, 8 up, 8 in 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4054888578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi174-79693-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: pgmap v138: 452 pgs: 96 creating+peering, 32 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s wr, 3 op/s 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: osdmap e126: 8 total, 8 up, 8 in 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4054888578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi174-79693-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: pgmap v138: 452 pgs: 96 creating+peering, 32 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s wr, 3 op/s 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi174-79705-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: osdmap e126: 8 total, 8 up, 8 in 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4054888578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi174-79693-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4054888578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi174-79693-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: osdmap e127: 8 total, 8 up, 8 in 2024-02-10T06:47:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:47:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:47:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: osdmap e128: 8 total, 8 up, 8 in 2024-02-10T06:47:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1724715133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4054888578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi174-79693-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: osdmap e127: 8 total, 8 up, 8 in 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: osdmap e128: 8 total, 8 up, 8 in 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1724715133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4054888578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi174-79693-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: osdmap e127: 8 total, 8 up, 8 in 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi174-79979-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: osdmap e128: 8 total, 8 up, 8 in 2024-02-10T06:47:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1724715133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: pgmap v141: 388 pgs: 32 creating+peering, 64 unknown, 292 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: osdmap e129: 8 total, 8 up, 8 in 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2080994475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi174-79693-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:31.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: pgmap v141: 388 pgs: 32 creating+peering, 64 unknown, 292 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: osdmap e129: 8 total, 8 up, 8 in 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2080994475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi174-79693-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: pgmap v141: 388 pgs: 32 creating+peering, 64 unknown, 292 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi174-79705-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: osdmap e129: 8 total, 8 up, 8 in 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2080994475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi174-79693-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: pgmap v144: 428 pgs: 37 creating+peering, 25 unknown, 366 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 93 KiB/s wr, 91 op/s 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2080994475' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi174-79693-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: osdmap e130: 8 total, 8 up, 8 in 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:47:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: pgmap v144: 428 pgs: 37 creating+peering, 25 unknown, 366 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 93 KiB/s wr, 91 op/s 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2080994475' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi174-79693-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: osdmap e130: 8 total, 8 up, 8 in 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:47:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: pgmap v144: 428 pgs: 37 creating+peering, 25 unknown, 366 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 93 KiB/s wr, 91 op/s 2024-02-10T06:47:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2080994475' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi174-79693-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:47:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: osdmap e130: 8 total, 8 up, 8 in 2024-02-10T06:47:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:47:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:47:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]': finished 2024-02-10T06:47:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:33 smithi184 ceph-mon[33513]: osdmap e131: 8 total, 8 up, 8 in 2024-02-10T06:47:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]: dispatch 2024-02-10T06:47:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2093878553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi174-79705-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]: dispatch 2024-02-10T06:47:33.488 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]': finished 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[39523]: osdmap e131: 8 total, 8 up, 8 in 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2093878553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi174-79705-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-13"}]': finished 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[29878]: osdmap e131: 8 total, 8 up, 8 in 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2093878553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi174-79705-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]: dispatch 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: pgmap v147: 396 pgs: 32 unknown, 364 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 376 KiB/s wr, 120 op/s 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2093878553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi174-79705-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]': finished 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: osdmap e132: 8 total, 8 up, 8 in 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3682641892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: pgmap v147: 396 pgs: 32 unknown, 364 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 376 KiB/s wr, 120 op/s 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2093878553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi174-79705-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]': finished 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: osdmap e132: 8 total, 8 up, 8 in 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3682641892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: pgmap v147: 396 pgs: 32 unknown, 364 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 376 KiB/s wr, 120 op/s 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2093878553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi174-79705-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-13", "mode": "writeback"}]': finished 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: osdmap e132: 8 total, 8 up, 8 in 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3682641892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-10T06:47:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: 172.5 scrub starts 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: 172.5 scrub ok 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: 172.8 scrub starts 2024-02-10T06:47:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: 172.8 scrub ok 2024-02-10T06:47:35.375 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:35.375 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:35 smithi184 ceph-mon[33513]: osdmap e133: 8 total, 8 up, 8 in 2024-02-10T06:47:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-10T06:47:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-10T06:47:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-10T06:47:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-10T06:47:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-10T06:47:35.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: 172.5 scrub starts 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: 172.5 scrub ok 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: 172.8 scrub starts 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: 172.8 scrub ok 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[29878]: osdmap e133: 8 total, 8 up, 8 in 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-10T06:47:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-10T06:47:35.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: 172.5 scrub starts 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: 172.5 scrub ok 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: 172.8 scrub starts 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: 172.8 scrub ok 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi174-79693-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:35.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:35 smithi174 ceph-mon[39523]: osdmap e133: 8 total, 8 up, 8 in 2024-02-10T06:47:36.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.9 scrub starts 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.9 scrub ok 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.4 scrub starts 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.4 scrub ok 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: pgmap v150: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.0 deep-scrub starts 2024-02-10T06:47:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.0 deep-scrub ok 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.7 scrub starts 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.7 scrub ok 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.3 scrub starts 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.3 scrub ok 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.1 scrub starts 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: 172.1 scrub ok 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: osdmap e134: 8 total, 8 up, 8 in 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3374724370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:36.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.9 scrub starts 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.9 scrub ok 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.4 scrub starts 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.4 scrub ok 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: pgmap v150: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.0 deep-scrub starts 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.0 deep-scrub ok 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.7 scrub starts 2024-02-10T06:47:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.7 scrub ok 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.3 scrub starts 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.3 scrub ok 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.1 scrub starts 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: 172.1 scrub ok 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: osdmap e134: 8 total, 8 up, 8 in 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3374724370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:36.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.9 scrub starts 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.9 scrub ok 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.4 scrub starts 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.4 scrub ok 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: pgmap v150: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.0 deep-scrub starts 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.0 deep-scrub ok 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.7 scrub starts 2024-02-10T06:47:36.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.7 scrub ok 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.3 scrub starts 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.3 scrub ok 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.1 scrub starts 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: 172.1 scrub ok 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: osdmap e134: 8 total, 8 up, 8 in 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3374724370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:36.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: 172.2 scrub starts 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: 172.2 scrub ok 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: 172.6 scrub starts 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: 172.6 scrub ok 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: osdmap e135: 8 total, 8 up, 8 in 2024-02-10T06:47:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1699523346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: 172.2 scrub starts 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: 172.2 scrub ok 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: 172.6 scrub starts 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: 172.6 scrub ok 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: osdmap e135: 8 total, 8 up, 8 in 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1699523346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: 172.2 scrub starts 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: 172.2 scrub ok 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: 172.6 scrub starts 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: 172.6 scrub ok 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi174-79705-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: osdmap e135: 8 total, 8 up, 8 in 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1699523346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:37.988 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:47:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:47:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:47:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:38 smithi184 ceph-mon[33513]: pgmap v153: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 61 KiB/s rd, 0 B/s wr, 84 op/s 2024-02-10T06:47:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:38 smithi184 ceph-mon[33513]: osdmap e136: 8 total, 8 up, 8 in 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[29878]: pgmap v153: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 61 KiB/s rd, 0 B/s wr, 84 op/s 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[29878]: osdmap e136: 8 total, 8 up, 8 in 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[39523]: pgmap v153: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 61 KiB/s rd, 0 B/s wr, 84 op/s 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi174-79693-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:38 smithi174 ceph-mon[39523]: osdmap e136: 8 total, 8 up, 8 in 2024-02-10T06:47:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:39 smithi184 ceph-mon[33513]: osdmap e137: 8 total, 8 up, 8 in 2024-02-10T06:47:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1802560385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi174-79705-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[29878]: osdmap e137: 8 total, 8 up, 8 in 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1802560385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi174-79705-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[39523]: osdmap e137: 8 total, 8 up, 8 in 2024-02-10T06:47:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1802560385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi174-79705-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: pgmap v156: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 82 op/s 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1802560385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi174-79705-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: osdmap e138: 8 total, 8 up, 8 in 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3936048703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: pgmap v156: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 82 op/s 2024-02-10T06:47:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1802560385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi174-79705-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: osdmap e138: 8 total, 8 up, 8 in 2024-02-10T06:47:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3936048703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: pgmap v156: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 82 op/s 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1802560385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi174-79705-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: osdmap e138: 8 total, 8 up, 8 in 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2578333427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3936048703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:40.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-10T06:47:40.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-02-10T06:47:40.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-10T06:47:40.850 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (3501 ms) 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (4989 ms) 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (4585 ms) 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2019 ms) 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (15094 ms total) 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5560 ms) 2024-02-10T06:47:40.851 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4650 ms) 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (6148 ms) 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (16358 ms total) 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3035 ms) 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1958 ms) 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2016 ms) 2024-02-10T06:47:40.852 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2011 ms) 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9020 ms total) 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4049 ms) 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4017 ms) 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8066 ms total) 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (73916 ms total) 2024-02-10T06:47:40.853 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: osdmap e139: 8 total, 8 up, 8 in 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2"}]: dispatch 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: osdmap e139: 8 total, 8 up, 8 in 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2"}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi174-79979-15"}]': finished 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi174-79693-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: osdmap e139: 8 total, 8 up, 8 in 2024-02-10T06:47:41.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2"}]: dispatch 2024-02-10T06:47:41.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:41.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: pgmap v159: 356 pgs: 11 unknown, 345 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: osdmap e140: 8 total, 8 up, 8 in 2024-02-10T06:47:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2930196029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3029713118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: pgmap v159: 356 pgs: 11 unknown, 345 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: osdmap e140: 8 total, 8 up, 8 in 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2930196029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3029713118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: pgmap v159: 356 pgs: 11 unknown, 345 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi174-79923-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: osdmap e140: 8 total, 8 up, 8 in 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2930196029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3029713118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:42.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: osdmap e141: 8 total, 8 up, 8 in 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2302591140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:47:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: osdmap e141: 8 total, 8 up, 8 in 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2302591140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:47:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79986-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: osdmap e141: 8 total, 8 up, 8 in 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2302591140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T06:47:43.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: pgmap v162: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: osdmap e142: 8 total, 8 up, 8 in 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:47:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: pgmap v162: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: osdmap e142: 8 total, 8 up, 8 in 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:47:44.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:47:44.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: pgmap v162: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T06:47:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79693-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: osdmap e142: 8 total, 8 up, 8 in 2024-02-10T06:47:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4281045609' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:47:44.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T06:47:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]': finished 2024-02-10T06:47:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[29878]: osdmap e143: 8 total, 8 up, 8 in 2024-02-10T06:47:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3777267255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]': finished 2024-02-10T06:47:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[39523]: osdmap e143: 8 total, 8 up, 8 in 2024-02-10T06:47:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3777267255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:45.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:45.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi174-79923-1","app":"app1","key":"key1"}]': finished 2024-02-10T06:47:45.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:45 smithi184 ceph-mon[33513]: osdmap e143: 8 total, 8 up, 8 in 2024-02-10T06:47:45.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3777267255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:45.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:45.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-10T06:47:45.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-10T06:47:45.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-10T06:47:45.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-10T06:47:45.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-10T06:47:45.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: seed 79923 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (177 ms) 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (10 ms) 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-10T06:47:45.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (6 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (5 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (1 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (3 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (5 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (1 ms) 2024-02-10T06:47:45.880 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (3 ms) 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (4 ms) 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1521 ms) 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (374 ms) 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: done waiting, doing copies 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-10T06:47:45.881 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: done waiting 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60815 ms) 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4498 ms) 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-10T06:47:45.882 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-10T06:47:46.375 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:47:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: pgmap v165: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:47:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: osdmap e144: 8 total, 8 up, 8 in 2024-02-10T06:47:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1726131071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: pgmap v165: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: osdmap e144: 8 total, 8 up, 8 in 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1726131071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: pgmap v165: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi174-79705-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: osdmap e144: 8 total, 8 up, 8 in 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1726131071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:46.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:47:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: osdmap e145: 8 total, 8 up, 8 in 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: osdmap e145: 8 total, 8 up, 8 in 2024-02-10T06:47:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:47.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:47.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:47.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi174-79693-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79923-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:47:47.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: osdmap e145: 8 total, 8 up, 8 in 2024-02-10T06:47:47.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:47.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:47.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:47.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:47:48.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:47:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:47:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:47:48.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:48 smithi184 ceph-mon[33513]: pgmap v168: 324 pgs: 32 creating+peering, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:48 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:48 smithi184 ceph-mon[33513]: osdmap e146: 8 total, 8 up, 8 in 2024-02-10T06:47:48.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/419172451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi174-79705-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[39523]: pgmap v168: 324 pgs: 32 creating+peering, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[39523]: osdmap e146: 8 total, 8 up, 8 in 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/419172451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi174-79705-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[29878]: pgmap v168: 324 pgs: 32 creating+peering, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[29878]: osdmap e146: 8 total, 8 up, 8 in 2024-02-10T06:47:48.990 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/419172451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi174-79705-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/419172451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi174-79705-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: osdmap e147: 8 total, 8 up, 8 in 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1581335112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/419172451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi174-79705-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: osdmap e147: 8 total, 8 up, 8 in 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1581335112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:49.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79923-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:49.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi174-79986-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:47:49.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/419172451' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi174-79705-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:49.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: osdmap e147: 8 total, 8 up, 8 in 2024-02-10T06:47:49.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1581335112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:49.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: pgmap v171: 372 pgs: 80 unknown, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: osdmap e148: 8 total, 8 up, 8 in 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: pgmap v171: 372 pgs: 80 unknown, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: osdmap e148: 8 total, 8 up, 8 in 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: pgmap v171: 372 pgs: 80 unknown, 292 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi174-79693-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: osdmap e148: 8 total, 8 up, 8 in 2024-02-10T06:47:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:47:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:52.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:52.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:51 smithi184 ceph-mon[33513]: osdmap e149: 8 total, 8 up, 8 in 2024-02-10T06:47:52.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3462859170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi174-79705-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[29878]: osdmap e149: 8 total, 8 up, 8 in 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3462859170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi174-79705-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[39523]: osdmap e149: 8 total, 8 up, 8 in 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3462859170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi174-79705-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:53.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: pgmap v174: 372 pgs: 28 creating+peering, 15 unknown, 329 active+clean; 592 KiB data, 756 MiB used, 714 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-02-10T06:47:53.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3462859170' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi174-79705-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:53.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:53.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: osdmap e150: 8 total, 8 up, 8 in 2024-02-10T06:47:53.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3623852586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:53.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:53.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: pgmap v174: 372 pgs: 28 creating+peering, 15 unknown, 329 active+clean; 592 KiB data, 756 MiB used, 714 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3462859170' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi174-79705-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: osdmap e150: 8 total, 8 up, 8 in 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3623852586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: pgmap v174: 372 pgs: 28 creating+peering, 15 unknown, 329 active+clean; 592 KiB data, 756 MiB used, 714 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-02-10T06:47:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3462859170' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi174-79705-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79923-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: osdmap e150: 8 total, 8 up, 8 in 2024-02-10T06:47:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3623852586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:53.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6 api_snapshots_pp: Running main() from gmock_main.cc 2024-02-10T06:47:53.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-02-10T06:47:53.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-10T06:47:53.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-10T06:47:53.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: seed 79986 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2959 ms) 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4587 ms) 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (1764 ms) 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2760 ms) 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4354 ms) 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (16424 ms total) 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-10T06:47:53.838 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (6738 ms) 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (7178 ms) 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6055 ms) 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3994 ms) 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2051 ms) 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1361-g290212a1/rpm/el9/BUILD/ceph-19.0.0-1361-g290212a1/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi174-79986-7 2024-02-10T06:47:53.839 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17950 ms) 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (43966 ms total) 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5075 ms) 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5084 ms total) 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2941 ms) 2024-02-10T06:47:53.840 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-10T06:47:54.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:54.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:53 smithi184 ceph-mon[33513]: osdmap e151: 8 total, 8 up, 8 in 2024-02-10T06:47:54.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:53 smithi174 ceph-mon[29878]: osdmap e151: 8 total, 8 up, 8 in 2024-02-10T06:47:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi174-79693-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:53 smithi174 ceph-mon[39523]: osdmap e151: 8 total, 8 up, 8 in 2024-02-10T06:47:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: pgmap v177: 340 pgs: 32 unknown, 308 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: osdmap e152: 8 total, 8 up, 8 in 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4209750451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:55.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: pgmap v177: 340 pgs: 32 unknown, 308 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:47:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: osdmap e152: 8 total, 8 up, 8 in 2024-02-10T06:47:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4209750451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: pgmap v177: 340 pgs: 32 unknown, 308 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:47:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:47:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: osdmap e152: 8 total, 8 up, 8 in 2024-02-10T06:47:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4209750451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:56.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: osdmap e153: 8 total, 8 up, 8 in 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2362573628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi174-79693-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:56.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: osdmap e153: 8 total, 8 up, 8 in 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2362573628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi174-79693-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:56.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi174-79705-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:56.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/152845702' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:56.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: osdmap e153: 8 total, 8 up, 8 in 2024-02-10T06:47:56.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]: dispatch 2024-02-10T06:47:56.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2362573628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi174-79693-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:56.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-mon[33513]: pgmap v180: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:47:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2362573628' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi174-79693-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-mon[33513]: osdmap e154: 8 total, 8 up, 8 in 2024-02-10T06:47:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[29878]: pgmap v180: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2362573628' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi174-79693-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[29878]: osdmap e154: 8 total, 8 up, 8 in 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[39523]: pgmap v180: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79923-24"}]': finished 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2362573628' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi174-79693-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[39523]: osdmap e154: 8 total, 8 up, 8 in 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:47:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [----------] Global test environment set-up. 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (15945 ms) 2024-02-10T06:47:57.875 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5520 ms) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3626 ms) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (6581 ms) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (8924 ms) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (40596 ms total) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (6165 ms) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5754 ms) 2024-02-10T06:47:57.876 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8037 ms) 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5978 ms) 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4019 ms) 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi174-80864-11. 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: Waiting for snaps to purge. 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20043 ms) 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49996 ms total) 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [----------] Global test environment tear-down 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (90592 ms total) 2024-02-10T06:47:57.877 INFO:tasks.workunit.client.0.smithi174.stdout: snapshots: [ PASSED ] 11 tests. 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: osdmap e155: 8 total, 8 up, 8 in 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1757599130' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999810869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: pgmap v183: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:47:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:47:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: osdmap e155: 8 total, 8 up, 8 in 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1757599130' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999810869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: pgmap v183: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:47:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: osdmap e155: 8 total, 8 up, 8 in 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1757599130' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999810869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: pgmap v183: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:47:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:58 smithi184 ceph-mon[33513]: osdmap e156: 8 total, 8 up, 8 in 2024-02-10T06:47:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3905270662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[29878]: osdmap e156: 8 total, 8 up, 8 in 2024-02-10T06:47:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3905270662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:47:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi174-79923-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi174-79705-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:47:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[39523]: osdmap e156: 8 total, 8 up, 8 in 2024-02-10T06:47:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3905270662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:47:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: osdmap e157: 8 total, 8 up, 8 in 2024-02-10T06:48:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: pgmap v186: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[29878]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: osdmap e157: 8 total, 8 up, 8 in 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: pgmap v186: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:47:59 smithi174 ceph-mon[39523]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi174-79693-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: osdmap e157: 8 total, 8 up, 8 in 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: pgmap v186: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:47:59 smithi184 ceph-mon[33513]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: osdmap e158: 8 total, 8 up, 8 in 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2687248688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2005622867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: osdmap e158: 8 total, 8 up, 8 in 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2687248688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2005622867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: osdmap e158: 8 total, 8 up, 8 in 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/838317038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2687248688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2005622867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: osdmap e159: 8 total, 8 up, 8 in 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2611744645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: pgmap v189: 356 pgs: 28 creating+peering, 1 active, 15 creating+activating, 16 unknown, 296 active+clean; 596 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: osdmap e159: 8 total, 8 up, 8 in 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2611744645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: pgmap v189: 356 pgs: 28 creating+peering, 1 active, 15 creating+activating, 16 unknown, 296 active+clean; 596 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:02.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi174-79986-16"}]': finished 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi174-79923-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi174-79705-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: osdmap e159: 8 total, 8 up, 8 in 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2611744645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: pgmap v189: 356 pgs: 28 creating+peering, 1 active, 15 creating+activating, 16 unknown, 296 active+clean; 596 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[29878]: osdmap e160: 8 total, 8 up, 8 in 2024-02-10T06:48:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[39523]: osdmap e160: 8 total, 8 up, 8 in 2024-02-10T06:48:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi174-79693-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:03 smithi184 ceph-mon[33513]: osdmap e160: 8 total, 8 up, 8 in 2024-02-10T06:48:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: pgmap v192: 324 pgs: 64 unknown, 260 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: osdmap e161: 8 total, 8 up, 8 in 2024-02-10T06:48:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1183071986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3909220364' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: osdmap e162: 8 total, 8 up, 8 in 2024-02-10T06:48:04.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: pgmap v192: 324 pgs: 64 unknown, 260 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: osdmap e161: 8 total, 8 up, 8 in 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1183071986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3909220364' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: osdmap e162: 8 total, 8 up, 8 in 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: pgmap v192: 324 pgs: 64 unknown, 260 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: osdmap e161: 8 total, 8 up, 8 in 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1183071986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3909220364' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi174-79705-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi174-79923-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi174-79693-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: osdmap e162: 8 total, 8 up, 8 in 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:04.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:05 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:05 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:05.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:05 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:06.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:48:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: pgmap v195: 268 pgs: 8 unknown, 260 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: osdmap e163: 8 total, 8 up, 8 in 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: osdmap e164: 8 total, 8 up, 8 in 2024-02-10T06:48:06.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi174-79923-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.375 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3853129805' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:06.375 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: pgmap v195: 268 pgs: 8 unknown, 260 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: osdmap e163: 8 total, 8 up, 8 in 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:06.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:06.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: osdmap e164: 8 total, 8 up, 8 in 2024-02-10T06:48:06.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi174-79923-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3853129805' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:06.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: pgmap v195: 268 pgs: 8 unknown, 260 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: osdmap e163: 8 total, 8 up, 8 in 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi174-79693-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: osdmap e164: 8 total, 8 up, 8 in 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi174-79923-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3853129805' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:06.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:07.598 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:07.599 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:07.599 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi174-79705-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:08.217 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:48:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:48:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: pgmap v198: 308 pgs: 39 creating+peering, 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 1 unknown, 226 active+clean; 606 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: osdmap e165: 8 total, 8 up, 8 in 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi174-79923-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: osdmap e166: 8 total, 8 up, 8 in 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: pgmap v198: 308 pgs: 39 creating+peering, 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 1 unknown, 226 active+clean; 606 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T06:48:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: osdmap e165: 8 total, 8 up, 8 in 2024-02-10T06:48:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi174-79923-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: osdmap e166: 8 total, 8 up, 8 in 2024-02-10T06:48:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: pgmap v198: 308 pgs: 39 creating+peering, 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 1 unknown, 226 active+clean; 606 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: osdmap e165: 8 total, 8 up, 8 in 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi174-79923-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: osdmap e166: 8 total, 8 up, 8 in 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: osdmap e167: 8 total, 8 up, 8 in 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/626642595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: osdmap e167: 8 total, 8 up, 8 in 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/626642595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:09.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:48:09.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:48:09.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: osdmap e167: 8 total, 8 up, 8 in 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/626642595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3999903473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]: dispatch 2024-02-10T06:48:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]: dispatch 2024-02-10T06:48:09.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: Running main() from gmock_main.cc 2024-02-10T06:48:09.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-02-10T06:48:09.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] Global test environment set-up. 2024-02-10T06:48:09.804 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.TooBig (9134 ms) 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3796 ms) 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (7001 ms) 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3574 ms) 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3075 ms) 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (4828 ms) 2024-02-10T06:48:09.805 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (5226 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (4466 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3880 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3018 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3038 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2972 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3012 ms) 2024-02-10T06:48:09.806 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3000 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.Flush (2972 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3009 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3020 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3016 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3009 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3013 ms) 2024-02-10T06:48:09.807 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3494 ms) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (2460 ms) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3033 ms) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3006 ms) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3040 ms) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3252 ms) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 26 tests from LibRadosAio (96344 ms total) 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: 2024-02-10T06:48:09.808 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-10T06:48:09.809 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: pgmap v201: 308 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 40 unknown, 226 active+clean; 606 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]': finished 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: osdmap e168: 8 total, 8 up, 8 in 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi174-79693-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[29878]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: pgmap v201: 308 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 40 unknown, 226 active+clean; 606 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]': finished 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: osdmap e168: 8 total, 8 up, 8 in 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:10.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:10.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi174-79693-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:10.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:10 smithi174 ceph-mon[39523]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: pgmap v201: 308 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 40 unknown, 226 active+clean; 606 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi174-79705-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-13"}]': finished 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi174-79693-27"}]': finished 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: osdmap e168: 8 total, 8 up, 8 in 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:10.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:10.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi174-79693-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:10.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:10 smithi184 ceph-mon[33513]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:11.656 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:11.656 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi174-79693-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:11.656 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:11 smithi184 ceph-mon[33513]: osdmap e169: 8 total, 8 up, 8 in 2024-02-10T06:48:11.656 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:11.656 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi174-79693-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi174-79693-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[29878]: osdmap e169: 8 total, 8 up, 8 in 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi174-79693-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi174-79693-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[39523]: osdmap e169: 8 total, 8 up, 8 in 2024-02-10T06:48:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]: dispatch 2024-02-10T06:48:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi174-79693-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:12.057 INFO:tasks.workunit.client.0.smithi174.stdout: 7435 ms total) 2024-02-10T06:48:12.057 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:48:12.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-10T06:48:12.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-10T06:48:12.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (118 ms) 2024-02-10T06:48:12.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (118 ms total) 2024-02-10T06:48:12.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:48:12.058 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (35 ms) 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (11 ms) 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (46 ms total) 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:48:12.059 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (38 ms) 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (5 ms) 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (43 ms total) 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-10T06:48:12.060 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (12 ms) 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (28 ms) 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (40 ms total) 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-10T06:48:12.061 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-10T06:48:12.062 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1021 ms) 2024-02-10T06:48:12.062 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1021 ms total) 2024-02-10T06:48:12.062 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: 2024-02-10T06:48:12.062 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-10T06:48:12.062 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (105151 ms total) 2024-02-10T06:48:12.062 INFO:tasks.workunit.client.0.smithi174.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: pgmap v204: 236 pgs: 2 active+clean+snaptrim_wait, 234 active+clean; 610 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: osdmap e170: 8 total, 8 up, 8 in 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/172336949' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: pgmap v204: 236 pgs: 2 active+clean+snaptrim_wait, 234 active+clean; 610 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: osdmap e170: 8 total, 8 up, 8 in 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/172336949' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: pgmap v204: 236 pgs: 2 active+clean+snaptrim_wait, 234 active+clean; 610 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-10T06:48:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3608659098' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi174-79923-36"}]': finished 2024-02-10T06:48:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: osdmap e170: 8 total, 8 up, 8 in 2024-02-10T06:48:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/172336949' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:12.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi174-79693-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:13 smithi184 ceph-mon[33513]: osdmap e171: 8 total, 8 up, 8 in 2024-02-10T06:48:13.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi174-79693-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[29878]: osdmap e171: 8 total, 8 up, 8 in 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi174-79693-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi174-79705-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:13 smithi174 ceph-mon[39523]: osdmap e171: 8 total, 8 up, 8 in 2024-02-10T06:48:14.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:14 smithi184 ceph-mon[33513]: pgmap v207: 308 pgs: 72 unknown, 236 active+clean; 609 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-10T06:48:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:14 smithi184 ceph-mon[33513]: osdmap e172: 8 total, 8 up, 8 in 2024-02-10T06:48:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[39523]: pgmap v207: 308 pgs: 72 unknown, 236 active+clean; 609 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[39523]: osdmap e172: 8 total, 8 up, 8 in 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[29878]: pgmap v207: 308 pgs: 72 unknown, 236 active+clean; 609 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[29878]: osdmap e172: 8 total, 8 up, 8 in 2024-02-10T06:48:14.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:14.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: osdmap e173: 8 total, 8 up, 8 in 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/709226536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: osdmap e173: 8 total, 8 up, 8 in 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/709226536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: osdmap e173: 8 total, 8 up, 8 in 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/709226536' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:15.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:16.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: pgmap v210: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]': finished 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: osdmap e174: 8 total, 8 up, 8 in 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]: dispatch 2024-02-10T06:48:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]: dispatch 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: pgmap v210: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]': finished 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: osdmap e174: 8 total, 8 up, 8 in 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]: dispatch 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]: dispatch 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: pgmap v210: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-15"}]': finished 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi174-79705-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: osdmap e174: 8 total, 8 up, 8 in 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]: dispatch 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]: dispatch 2024-02-10T06:48:16.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]: dispatch 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]': finished 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: osdmap e175: 8 total, 8 up, 8 in 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi174-79693-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]': finished 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: osdmap e175: 8 total, 8 up, 8 in 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi174-79693-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4130410839' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi174-79693-28"}]': finished 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-15", "mode": "writeback"}]': finished 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: osdmap e175: 8 total, 8 up, 8 in 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:17.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi174-79693-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:48:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:48:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: pgmap v213: 260 pgs: 260 active+clean; 585 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi174-79693-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: osdmap e176: 8 total, 8 up, 8 in 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi174-79693-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1886078842' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi174-79705-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:18.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: pgmap v213: 260 pgs: 260 active+clean; 585 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi174-79693-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: osdmap e176: 8 total, 8 up, 8 in 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi174-79693-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1886078842' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi174-79705-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: pgmap v213: 260 pgs: 260 active+clean; 585 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi174-79693-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: osdmap e176: 8 total, 8 up, 8 in 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]: dispatch 2024-02-10T06:48:18.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi174-79693-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:18.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1886078842' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi174-79705-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:18.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:18.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout:napshotsECPP.SnapRemovePP (2018 ms) 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2014 ms) 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2012 ms) 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8985 ms total) 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4017 ms) 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-10T06:48:19.063 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (4113 ms) 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3760 ms) 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11890 ms total) 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (112146 ms total) 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-02-10T06:48:19.064 INFO:tasks.workunit.client.0.smithi174.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1886078842' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi174-79705-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]': finished 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: osdmap e177: 8 total, 8 up, 8 in 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T06:48:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T06:48:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1886078842' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi174-79705-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]': finished 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: osdmap e177: 8 total, 8 up, 8 in 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T06:48:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/622831143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi174-79986-21"}]': finished 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1886078842' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi174-79705-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]': finished 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: osdmap e177: 8 total, 8 up, 8 in 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]: dispatch 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T06:48:19.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-10T06:48:19.833 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (9130 ms) 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (22988 ms) 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (7946 ms) 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3885 ms) 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3285 ms) 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-10T06:48:19.834 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2792 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2967 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3010 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2999 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3269 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2727 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3004 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3023 ms) 2024-02-10T06:48:19.835 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3022 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3007 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3012 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2929 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3018 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3034 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3024 ms) 2024-02-10T06:48:19.836 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3022 ms) 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3260 ms) 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2761 ms) 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2887 ms) 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3015 ms) 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2992 ms) 2024-02-10T06:48:19.837 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: pgmap v216: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi174-79693-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]': finished 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: osdmap e178: 8 total, 8 up, 8 in 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:21.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:20 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: pgmap v216: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi174-79693-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]': finished 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: osdmap e178: 8 total, 8 up, 8 in 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: pgmap v216: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi174-79693-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-15"}]': finished 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: osdmap e178: 8 total, 8 up, 8 in 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:20 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:22.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:21 smithi184 ceph-mon[33513]: osdmap e179: 8 total, 8 up, 8 in 2024-02-10T06:48:22.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:22.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/337216158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:22.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[29878]: osdmap e179: 8 total, 8 up, 8 in 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/337216158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[39523]: osdmap e179: 8 total, 8 up, 8 in 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/337216158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: pgmap v219: 268 pgs: 30 creating+peering, 3 unknown, 235 active+clean; 586 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: osdmap e180: 8 total, 8 up, 8 in 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: pgmap v219: 268 pgs: 30 creating+peering, 3 unknown, 235 active+clean; 586 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-02-10T06:48:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: osdmap e180: 8 total, 8 up, 8 in 2024-02-10T06:48:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: pgmap v219: 268 pgs: 30 creating+peering, 3 unknown, 235 active+clean; 586 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi174-79705-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: osdmap e180: 8 total, 8 up, 8 in 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: osdmap e181: 8 total, 8 up, 8 in 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: pgmap v222: 260 pgs: 32 unknown, 228 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: osdmap e182: 8 total, 8 up, 8 in 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/257668564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: osdmap e181: 8 total, 8 up, 8 in 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: pgmap v222: 260 pgs: 32 unknown, 228 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: osdmap e182: 8 total, 8 up, 8 in 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/257668564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: osdmap e181: 8 total, 8 up, 8 in 2024-02-10T06:48:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]: dispatch 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: pgmap v222: 260 pgs: 32 unknown, 228 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3717243908' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi174-79693-29"}]': finished 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:24.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: osdmap e182: 8 total, 8 up, 8 in 2024-02-10T06:48:24.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:24.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:24.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/257668564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:24.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]': finished 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: osdmap e183: 8 total, 8 up, 8 in 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]: dispatch 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]: dispatch 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]': finished 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:25.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: osdmap e183: 8 total, 8 up, 8 in 2024-02-10T06:48:25.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]: dispatch 2024-02-10T06:48:25.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]: dispatch 2024-02-10T06:48:25.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:25.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-17"}]': finished 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi174-79705-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:25.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi174-79693-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:25.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: osdmap e183: 8 total, 8 up, 8 in 2024-02-10T06:48:25.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]: dispatch 2024-02-10T06:48:25.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]: dispatch 2024-02-10T06:48:25.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:25.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:26 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]': finished 2024-02-10T06:48:26.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:48:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:48:26.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:26 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]': finished 2024-02-10T06:48:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:26 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-17", "mode": "writeback"}]': finished 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: pgmap v225: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: osdmap e184: 8 total, 8 up, 8 in 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: osdmap e185: 8 total, 8 up, 8 in 2024-02-10T06:48:27.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:27.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3869792076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi174-79705-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:27.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:27.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:27.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: pgmap v225: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: osdmap e184: 8 total, 8 up, 8 in 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: osdmap e185: 8 total, 8 up, 8 in 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3869792076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi174-79705-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:27.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: pgmap v225: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: osdmap e184: 8 total, 8 up, 8 in 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi174-79693-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: osdmap e185: 8 total, 8 up, 8 in 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3869792076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi174-79705-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]: dispatch 2024-02-10T06:48:27.413 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:28.069 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:48:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:48:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[29878]: pgmap v228: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3869792076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi174-79705-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]': finished 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[29878]: osdmap e186: 8 total, 8 up, 8 in 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[39523]: pgmap v228: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:28.360 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3869792076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi174-79705-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:28.361 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]': finished 2024-02-10T06:48:28.361 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[39523]: osdmap e186: 8 total, 8 up, 8 in 2024-02-10T06:48:28.361 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:28 smithi184 ceph-mon[33513]: pgmap v228: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T06:48:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:28 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3869792076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi174-79705-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-17"}]': finished 2024-02-10T06:48:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:28 smithi184 ceph-mon[33513]: osdmap e186: 8 total, 8 up, 8 in 2024-02-10T06:48:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:30.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: osdmap e187: 8 total, 8 up, 8 in 2024-02-10T06:48:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T06:48:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:48:30.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:29 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: osdmap e187: 8 total, 8 up, 8 in 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: osdmap e187: 8 total, 8 up, 8 in 2024-02-10T06:48:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T06:48:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:48:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:29 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:31.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: pgmap v231: 228 pgs: 228 active+clean; 585 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: osdmap e188: 8 total, 8 up, 8 in 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3419526974' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:48:31.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:48:31.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: pgmap v231: 228 pgs: 228 active+clean; 585 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:48:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T06:48:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: osdmap e188: 8 total, 8 up, 8 in 2024-02-10T06:48:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3419526974' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: pgmap v231: 228 pgs: 228 active+clean; 585 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: osdmap e188: 8 total, 8 up, 8 in 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4179038477' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3419526974' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:48:31.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:32.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:32.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:32.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:32.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: osdmap e189: 8 total, 8 up, 8 in 2024-02-10T06:48:32.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:32.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:32.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi174-79693-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3419526974' entity='client.admin' cmd=[{ 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "prefix": "osd pool set", 2024-02-10T06:48:32.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "var": "eio", 2024-02-10T06:48:32.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "val": "true" 2024-02-10T06:48:32.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: }]: dispatch 2024-02-10T06:48:32.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-10T06:48:32.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "prefix": "osd pool set", 2024-02-10T06:48:32.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.127 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "var": "eio", 2024-02-10T06:48:32.127 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "val": "true" 2024-02-10T06:48:32.127 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: }]: dispatch 2024-02-10T06:48:32.127 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:32.127 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi174-79693-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:32.128 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]': finished 2024-02-10T06:48:32.128 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-10T06:48:32.128 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-10T06:48:32.128 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "prefix": "osd pool set", 2024-02-10T06:48:32.128 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "var": "eio", 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: "val": "true" 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: }]': finished 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: osdmap e190: 8 total, 8 up, 8 in 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi174-79693-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T06:48:32.129 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: osdmap e189: 8 total, 8 up, 8 in 2024-02-10T06:48:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi174-79693-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3419526974' entity='client.admin' cmd=[{ 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "prefix": "osd pool set", 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "var": "eio", 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "val": "true" 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: }]: dispatch 2024-02-10T06:48:32.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "prefix": "osd pool set", 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "var": "eio", 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "val": "true" 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: }]: dispatch 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi174-79693-30"}]': finished 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi174-79705-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:32.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: osdmap e189: 8 total, 8 up, 8 in 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi174-79693-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3419526974' entity='client.admin' cmd=[{ 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "prefix": "osd pool set", 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "var": "eio", 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "val": "true" 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: }]: dispatch 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-10T06:48:32.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "prefix": "osd pool set", 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "var": "eio", 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "val": "true" 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: }]: dispatch 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi174-79693-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]': finished 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "prefix": "osd pool set", 2024-02-10T06:48:32.243 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "var": "eio", 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: "val": "true" 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: }]': finished 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: osdmap e190: 8 total, 8 up, 8 in 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi174-79693-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi174-79693-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pg_num","val":"11"}]': finished 2024-02-10T06:48:32.244 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "prefix": "osd pool set", 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "pool": "PoolEIOFlag_smithi174-79705-33", 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "var": "eio", 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: "val": "true" 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: }]': finished 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: osdmap e190: 8 total, 8 up, 8 in 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi174-79693-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T06:48:32.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:33.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:33 smithi184 ceph-mon[33513]: pgmap v234: 292 pgs: 6 unknown, 286 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T06:48:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T06:48:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:33 smithi174 ceph-mon[29878]: pgmap v234: 292 pgs: 6 unknown, 286 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T06:48:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T06:48:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:33 smithi174 ceph-mon[39523]: pgmap v234: 292 pgs: 6 unknown, 286 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s wr, 7 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T06:48:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi174-79779-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T06:48:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]': finished 2024-02-10T06:48:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]: dispatch 2024-02-10T06:48:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: pgmap v237: 260 pgs: 260 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T06:48:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: osdmap e191: 8 total, 8 up, 8 in 2024-02-10T06:48:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]: dispatch 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi174-79693-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]': finished 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: osdmap e192: 8 total, 8 up, 8 in 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3151814058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]': finished 2024-02-10T06:48:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]: dispatch 2024-02-10T06:48:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: pgmap v237: 260 pgs: 260 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T06:48:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: osdmap e191: 8 total, 8 up, 8 in 2024-02-10T06:48:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]: dispatch 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi174-79693-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]': finished 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: osdmap e192: 8 total, 8 up, 8 in 2024-02-10T06:48:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3151814058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-19"}]': finished 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]: dispatch 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: pgmap v237: 260 pgs: 260 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T06:48:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: osdmap e191: 8 total, 8 up, 8 in 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]: dispatch 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi174-79693-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-19", "mode": "writeback"}]': finished 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi174-79779-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: osdmap e192: 8 total, 8 up, 8 in 2024-02-10T06:48:34.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3151814058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:34.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:34.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:34.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: osdmap e193: 8 total, 8 up, 8 in 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: osdmap e193: 8 total, 8 up, 8 in 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi174-79705-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: osdmap e193: 8 total, 8 up, 8 in 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]: dispatch 2024-02-10T06:48:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1942803866' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T06:48:36.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: pgmap v240: 299 pgs: 40 unknown, 259 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]': finished 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: osdmap e194: 8 total, 8 up, 8 in 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:36.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi174-79779-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: pgmap v240: 299 pgs: 40 unknown, 259 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]': finished 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: osdmap e194: 8 total, 8 up, 8 in 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:36.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi174-79779-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: pgmap v240: 299 pgs: 40 unknown, 259 active+clean; 585 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-19"}]': finished 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: osdmap e194: 8 total, 8 up, 8 in 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:36.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi174-79779-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi174-79779-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: osdmap e195: 8 total, 8 up, 8 in 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi174-79779-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/138302797' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:48:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:48:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi174-79779-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: osdmap e195: 8 total, 8 up, 8 in 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi174-79779-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/138302797' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi174-79779-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: osdmap e195: 8 total, 8 up, 8 in 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]: dispatch 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi174-79779-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/138302797' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:38.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: pgmap v243: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: osdmap e196: 8 total, 8 up, 8 in 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:39.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: pgmap v243: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: osdmap e196: 8 total, 8 up, 8 in 2024-02-10T06:48:39.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:39.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:39.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: pgmap v243: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:39.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2153719851' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi174-79693-31"}]': finished 2024-02-10T06:48:39.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi174-79705-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: osdmap e196: 8 total, 8 up, 8 in 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:39.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:39.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:39.920 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAiontry=6 expected=6 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:e960b815:::9:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:e960b815:::9:head expected=18:e960b815:::9:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:e960b815:::9:head -> 9 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=9 expected=9 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:de5d7c5f:::12:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:de5d7c5f:::12:head expected=18:de5d7c5f:::12:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:de5d7c5f:::12:head -> 12 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=12 expected=12 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:d83876eb:::4:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:d83876eb:::4:head expected=18:d83876eb:::4:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:d83876eb:::4:head -> 4 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=4 expected=4 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:bd63b0f1:::8:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:bd63b0f1:::8:head expected=18:bd63b0f1:::8:head 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:bd63b0f1:::8:head -> 8 2024-02-10T06:48:39.921 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=8 expected=8 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:02547ec2:::1:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:02547ec2:::1:head expected=18:02547ec2:::1:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:02547ec2:::1:head -> 1 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=1 expected=1 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:f905c69b:::2:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:f905c69b:::2:head expected=18:f905c69b:::2:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:f905c69b:::2:head -> 2 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=2 expected=2 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:62a1935d:::14:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:62a1935d:::14:head expected=18:62a1935d:::14:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:62a1935d:::14:head -> 14 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=14 expected=14 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : seek to 18:566253c9:::13:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : cursor()=18:566253c9:::13:head expected=18:566253c9:::13:head 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: > 18:566253c9:::13:head -> 13 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: : entry=13 expected=13 2024-02-10T06:48:39.922 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (192 ms) 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (113929 ms) 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 0/5 -> MIN 18:33333333::::head 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 1/5 -> 18:33333333::::head 18:66666666::::head 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 2/5 -> 18:66666666::::head 18:99999999::::head 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 3/5 -> 18:99999999::::head 18:cccccccc::::head 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: split 4/5 -> 18:cccccccc::::head MAX 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (4236 ms) 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 7 tests from LibRadosList (119192 ms total) 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-10T06:48:39.923 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1032 ms) 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo2,foo3 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo2 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo3 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo1,foo4,foo5 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo4 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo5 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo1 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset foo6,foo7 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo7 2024-02-10T06:48:39.924 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: foo6 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo4 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo5 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo7 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns2:foo6 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: ns1:foo1 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo1 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo2 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: :foo3 2024-02-10T06:48:39.925 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (30 ms) 2024-02-10T06:48:40.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi174-79779-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: osdmap e197: 8 total, 8 up, 8 in 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:40.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:39 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi174-79779-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: osdmap e197: 8 total, 8 up, 8 in 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:40.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi174-79779-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi174-79693-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: osdmap e197: 8 total, 8 up, 8 in 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:39 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:40 smithi184 ceph-mon[33513]: pgmap v246: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:40 smithi184 ceph-mon[33513]: osdmap e198: 8 total, 8 up, 8 in 2024-02-10T06:48:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2515094039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[29878]: pgmap v246: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[29878]: osdmap e198: 8 total, 8 up, 8 in 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2515094039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[39523]: pgmap v246: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[39523]: osdmap e198: 8 total, 8 up, 8 in 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:41.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2515094039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]': finished 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2515094039' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: osdmap e199: 8 total, 8 up, 8 in 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]: dispatch 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]: dispatch 2024-02-10T06:48:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[29878]: pgmap v249: 268 pgs: 7 creating+peering, 9 unknown, 252 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]': finished 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2515094039' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: osdmap e199: 8 total, 8 up, 8 in 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]: dispatch 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]: dispatch 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:41 smithi174 ceph-mon[39523]: pgmap v249: 268 pgs: 7 creating+peering, 9 unknown, 252 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi174-79693-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-21"}]': finished 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2515094039' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: osdmap e199: 8 total, 8 up, 8 in 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]: dispatch 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]: dispatch 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:41 smithi184 ceph-mon[33513]: pgmap v249: 268 pgs: 7 creating+peering, 9 unknown, 252 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]': finished 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: osdmap e200: 8 total, 8 up, 8 in 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1419900681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]': finished 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: osdmap e200: 8 total, 8 up, 8 in 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1419900681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-21", "mode": "writeback"}]': finished 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: osdmap e200: 8 total, 8 up, 8 in 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]: dispatch 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1419900681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1419900681' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: osdmap e201: 8 total, 8 up, 8 in 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[29878]: pgmap v252: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1419900681' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: osdmap e201: 8 total, 8 up, 8 in 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:43 smithi174 ceph-mon[39523]: pgmap v252: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:48:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228123199' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi174-79779-2"}]': finished 2024-02-10T06:48:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1419900681' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: osdmap e201: 8 total, 8 up, 8 in 2024-02-10T06:48:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:44.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]: dispatch 2024-02-10T06:48:44.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:44.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:43 smithi184 ceph-mon[33513]: pgmap v252: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]': finished 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: osdmap e202: 8 total, 8 up, 8 in 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3412071309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]': finished 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: osdmap e202: 8 total, 8 up, 8 in 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3412071309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:44 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-21"}]': finished 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: osdmap e202: 8 total, 8 up, 8 in 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2996343554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]: dispatch 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3412071309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:44 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:45.937 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 1 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 10 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 13 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 7 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 14 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 0 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 15 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 11 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 5 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 8 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 6 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 3 0 2024-02-10T06:48:45.938 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 4 0 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 12 0 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 9 0 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2 0 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (44 ms) 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 3 tests from LibRadosListEC (1106 ms total) 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3037 ms) 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] 1 test from LibRadosListNP (3037 ms total) 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [----------] Global test environment tear-down 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [==========] 11 tests from 3 test suites ran. (139075 ms total) 2024-02-10T06:48:45.939 INFO:tasks.workunit.client.0.smithi174.stdout: api_list: [ PASSED ] 11 tests. 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3412071309' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: osdmap e203: 8 total, 8 up, 8 in 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: pgmap v255: 324 pgs: 96 unknown, 228 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi174-79779-3","pool2":"test-rados-api-smithi174-79779-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3412071309' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: osdmap e203: 8 total, 8 up, 8 in 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: pgmap v255: 324 pgs: 96 unknown, 228 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi174-79779-3","pool2":"test-rados-api-smithi174-79779-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:46.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:46.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi174-79693-32"}]': finished 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3412071309' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi174-79705-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: osdmap e203: 8 total, 8 up, 8 in 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: pgmap v255: 324 pgs: 96 unknown, 228 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi174-79779-3","pool2":"test-rados-api-smithi174-79779-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi174-79779-3","pool2":"test-rados-api-smithi174-79779-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T06:48:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: osdmap e204: 8 total, 8 up, 8 in 2024-02-10T06:48:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi174-79779-3","pool2":"test-rados-api-smithi174-79779-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: osdmap e204: 8 total, 8 up, 8 in 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:47.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:47.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:47.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/341594710' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi174-79779-3","pool2":"test-rados-api-smithi174-79779-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi174-79693-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: osdmap e204: 8 total, 8 up, 8 in 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:48:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:48:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-mon[29878]: pgmap v258: 260 pgs: 26 creating+peering, 6 unknown, 228 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:48:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-mon[29878]: osdmap e205: 8 total, 8 up, 8 in 2024-02-10T06:48:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-mon[39523]: pgmap v258: 260 pgs: 26 creating+peering, 6 unknown, 228 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:48:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:47 smithi174 ceph-mon[39523]: osdmap e205: 8 total, 8 up, 8 in 2024-02-10T06:48:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:47 smithi184 ceph-mon[33513]: pgmap v258: 260 pgs: 26 creating+peering, 6 unknown, 228 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:48:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:47 smithi184 ceph-mon[33513]: osdmap e205: 8 total, 8 up, 8 in 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[39523]: osdmap e206: 8 total, 8 up, 8 in 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[29878]: osdmap e206: 8 total, 8 up, 8 in 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi174-79693-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:48 smithi184 ceph-mon[33513]: osdmap e206: 8 total, 8 up, 8 in 2024-02-10T06:48:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[29878]: osdmap e207: 8 total, 8 up, 8 in 2024-02-10T06:48:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2636410584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[29878]: pgmap v261: 268 pgs: 26 creating+peering, 46 unknown, 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[39523]: osdmap e207: 8 total, 8 up, 8 in 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2636410584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:49 smithi174 ceph-mon[39523]: pgmap v261: 268 pgs: 26 creating+peering, 46 unknown, 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:48:50.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:48:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:49 smithi184 ceph-mon[33513]: osdmap e207: 8 total, 8 up, 8 in 2024-02-10T06:48:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2636410584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:49 smithi184 ceph-mon[33513]: pgmap v261: 268 pgs: 26 creating+peering, 46 unknown, 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2636410584' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]': finished 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: osdmap e208: 8 total, 8 up, 8 in 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]: dispatch 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]: dispatch 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]': finished 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2636410584' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]': finished 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: osdmap e208: 8 total, 8 up, 8 in 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]: dispatch 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]: dispatch 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:51.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]': finished 2024-02-10T06:48:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2636410584' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-23"}]': finished 2024-02-10T06:48:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: osdmap e208: 8 total, 8 up, 8 in 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]: dispatch 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]: dispatch 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:48:51.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:48:51.375 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-23", "mode": "writeback"}]': finished 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: osdmap e209: 8 total, 8 up, 8 in 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1646456792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: pgmap v264: 292 pgs: 3 creating+peering, 35 unknown, 254 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[29878]: osdmap e210: 8 total, 8 up, 8 in 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: osdmap e209: 8 total, 8 up, 8 in 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1646456792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: pgmap v264: 292 pgs: 3 creating+peering, 35 unknown, 254 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:51 smithi174 ceph-mon[39523]: osdmap e210: 8 total, 8 up, 8 in 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/759493202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: osdmap e209: 8 total, 8 up, 8 in 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1646456792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]: dispatch 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: pgmap v264: 292 pgs: 3 creating+peering, 35 unknown, 254 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s wr, 1 op/s 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi174-79693-33"}]': finished 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:51 smithi184 ceph-mon[33513]: osdmap e210: 8 total, 8 up, 8 in 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: osdmap e211: 8 total, 8 up, 8 in 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2548897256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:53.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: osdmap e211: 8 total, 8 up, 8 in 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2548897256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:48:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi174-79693-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: osdmap e211: 8 total, 8 up, 8 in 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2548897256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]: dispatch 2024-02-10T06:48:53.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:54 smithi174 ceph-mon[29878]: pgmap v266: 292 pgs: 4 creating+peering, 28 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T06:48:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:54 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2548897256' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:54 smithi174 ceph-mon[39523]: pgmap v266: 292 pgs: 4 creating+peering, 28 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T06:48:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:54 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2548897256' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:54.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:54 smithi184 ceph-mon[33513]: pgmap v266: 292 pgs: 4 creating+peering, 28 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T06:48:54.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:54 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:48:54.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2548897256' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]': finished 2024-02-10T06:48:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[29878]: osdmap e212: 8 total, 8 up, 8 in 2024-02-10T06:48:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:48:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[29878]: osdmap e213: 8 total, 8 up, 8 in 2024-02-10T06:48:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1182801023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]': finished 2024-02-10T06:48:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[39523]: osdmap e212: 8 total, 8 up, 8 in 2024-02-10T06:48:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:48:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[39523]: osdmap e213: 8 total, 8 up, 8 in 2024-02-10T06:48:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1182801023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-23"}]': finished 2024-02-10T06:48:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:55 smithi184 ceph-mon[33513]: osdmap e212: 8 total, 8 up, 8 in 2024-02-10T06:48:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi174-79693-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:48:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:55 smithi184 ceph-mon[33513]: osdmap e213: 8 total, 8 up, 8 in 2024-02-10T06:48:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1182801023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:56.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:48:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:48:57.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:56 smithi184 ceph-mon[33513]: pgmap v269: 324 pgs: 4 creating+peering, 60 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-02-10T06:48:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:56 smithi184 ceph-mon[33513]: osdmap e214: 8 total, 8 up, 8 in 2024-02-10T06:48:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:57.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[29878]: pgmap v269: 324 pgs: 4 creating+peering, 60 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[29878]: osdmap e214: 8 total, 8 up, 8 in 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[39523]: pgmap v269: 324 pgs: 4 creating+peering, 60 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[39523]: osdmap e214: 8 total, 8 up, 8 in 2024-02-10T06:48:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:57.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:57 smithi184 ceph-mon[33513]: osdmap e215: 8 total, 8 up, 8 in 2024-02-10T06:48:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4138706346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:58.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:48:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[29878]: osdmap e215: 8 total, 8 up, 8 in 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4138706346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:48:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[39523]: osdmap e215: 8 total, 8 up, 8 in 2024-02-10T06:48:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4138706346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:48:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: pgmap v272: 364 pgs: 32 unknown, 40 creating+peering, 292 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4138706346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: osdmap e216: 8 total, 8 up, 8 in 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:59.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: pgmap v272: 364 pgs: 32 unknown, 40 creating+peering, 292 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4138706346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: osdmap e216: 8 total, 8 up, 8 in 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: pgmap v272: 364 pgs: 32 unknown, 40 creating+peering, 292 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4138706346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi174-79705-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2537066270' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: osdmap e216: 8 total, 8 up, 8 in 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]: dispatch 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:48:59.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: pgmap v275: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: osdmap e217: 8 total, 8 up, 8 in 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi174-79693-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]': finished 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi174-79693-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: osdmap e218: 8 total, 8 up, 8 in 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi174-79693-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]: dispatch 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]: dispatch 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: pgmap v275: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: osdmap e217: 8 total, 8 up, 8 in 2024-02-10T06:49:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi174-79693-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]': finished 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi174-79693-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: osdmap e218: 8 total, 8 up, 8 in 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi174-79693-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]: dispatch 2024-02-10T06:49:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:48:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]: dispatch 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: pgmap v275: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi174-79693-34"}]': finished 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: osdmap e217: 8 total, 8 up, 8 in 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi174-79693-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-25"}]': finished 2024-02-10T06:49:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi174-79693-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: osdmap e218: 8 total, 8 up, 8 in 2024-02-10T06:49:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi174-79693-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]: dispatch 2024-02-10T06:49:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:48:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]: dispatch 2024-02-10T06:49:01.375 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:01 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:01.376 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]': finished 2024-02-10T06:49:01.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:01 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]': finished 2024-02-10T06:49:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:01 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-25", "mode": "writeback"}]': finished 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: pgmap v279: 292 pgs: 6 unknown, 286 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 257 B/s rd, 514 B/s wr, 1 op/s 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: osdmap e219: 8 total, 8 up, 8 in 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi174-79693-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: osdmap e220: 8 total, 8 up, 8 in 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: pgmap v279: 292 pgs: 6 unknown, 286 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 257 B/s rd, 514 B/s wr, 1 op/s 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: osdmap e219: 8 total, 8 up, 8 in 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi174-79693-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: osdmap e220: 8 total, 8 up, 8 in 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: pgmap v279: 292 pgs: 6 unknown, 286 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 257 B/s rd, 514 B/s wr, 1 op/s 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: osdmap e219: 8 total, 8 up, 8 in 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi174-79693-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: osdmap e220: 8 total, 8 up, 8 in 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]: dispatch 2024-02-10T06:49:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:03 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]': finished 2024-02-10T06:49:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:03 smithi184 ceph-mon[33513]: osdmap e221: 8 total, 8 up, 8 in 2024-02-10T06:49:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:03 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]': finished 2024-02-10T06:49:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:03 smithi174 ceph-mon[29878]: osdmap e221: 8 total, 8 up, 8 in 2024-02-10T06:49:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:03 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-25"}]': finished 2024-02-10T06:49:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:03 smithi174 ceph-mon[39523]: osdmap e221: 8 total, 8 up, 8 in 2024-02-10T06:49:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:04 smithi184 ceph-mon[33513]: pgmap v282: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:49:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:04 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:04 smithi184 ceph-mon[33513]: osdmap e222: 8 total, 8 up, 8 in 2024-02-10T06:49:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1474625847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[29878]: pgmap v282: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[29878]: osdmap e222: 8 total, 8 up, 8 in 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1474625847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[39523]: pgmap v282: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[39523]: osdmap e222: 8 total, 8 up, 8 in 2024-02-10T06:49:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1474625847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.111 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:06.111 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:06.111 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: osdmap e223: 8 total, 8 up, 8 in 2024-02-10T06:49:06.111 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:06.111 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.112 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.112 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:05 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: osdmap e223: 8 total, 8 up, 8 in 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi174-79705-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: osdmap e223: 8 total, 8 up, 8 in 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]: dispatch 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:05 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:06.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: pgmap v285: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: osdmap e224: 8 total, 8 up, 8 in 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:07.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: pgmap v285: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: osdmap e224: 8 total, 8 up, 8 in 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: pgmap v285: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4030271237' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi174-79693-35"}]': finished 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: osdmap e224: 8 total, 8 up, 8 in 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:07.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:07.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: osdmap e225: 8 total, 8 up, 8 in 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3227013205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:49:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: osdmap e225: 8 total, 8 up, 8 in 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3227013205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi174-79693-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: osdmap e225: 8 total, 8 up, 8 in 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3227013205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:08.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:09.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:08 smithi184 ceph-mon[33513]: pgmap v288: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-10T06:49:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:08 smithi184 ceph-mon[33513]: osdmap e226: 8 total, 8 up, 8 in 2024-02-10T06:49:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[29878]: pgmap v288: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[29878]: osdmap e226: 8 total, 8 up, 8 in 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[39523]: pgmap v288: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-10T06:49:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi174-79705-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[39523]: osdmap e226: 8 total, 8 up, 8 in 2024-02-10T06:49:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]': finished 2024-02-10T06:49:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: osdmap e227: 8 total, 8 up, 8 in 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: pgmap v291: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:49:10.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]': finished 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: osdmap e227: 8 total, 8 up, 8 in 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: pgmap v291: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:49:10.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:09 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi174-79693-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-27"}]': finished 2024-02-10T06:49:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: osdmap e227: 8 total, 8 up, 8 in 2024-02-10T06:49:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]: dispatch 2024-02-10T06:49:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: pgmap v291: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:49:10.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:09 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]': finished 2024-02-10T06:49:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[29878]: osdmap e228: 8 total, 8 up, 8 in 2024-02-10T06:49:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:11.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]': finished 2024-02-10T06:49:11.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:11.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[39523]: osdmap e228: 8 total, 8 up, 8 in 2024-02-10T06:49:11.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:11.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:11.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-27", "mode": "writeback"}]': finished 2024-02-10T06:49:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:10 smithi184 ceph-mon[33513]: osdmap e228: 8 total, 8 up, 8 in 2024-02-10T06:49:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[29878]: pgmap v294: 228 pgs: 228 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[29878]: osdmap e229: 8 total, 8 up, 8 in 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[39523]: pgmap v294: 228 pgs: 228 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:49:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[39523]: osdmap e229: 8 total, 8 up, 8 in 2024-02-10T06:49:12.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:12.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:12.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:12.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:11 smithi184 ceph-mon[33513]: pgmap v294: 228 pgs: 228 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:49:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:11 smithi184 ceph-mon[33513]: osdmap e229: 8 total, 8 up, 8 in 2024-02-10T06:49:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: osdmap e230: 8 total, 8 up, 8 in 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: osdmap e230: 8 total, 8 up, 8 in 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:13.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: osdmap e230: 8 total, 8 up, 8 in 2024-02-10T06:49:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2111733823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:13.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:13.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]: dispatch 2024-02-10T06:49:13.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]: dispatch 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]': finished 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: osdmap e231: 8 total, 8 up, 8 in 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[29878]: pgmap v297: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]': finished 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: osdmap e231: 8 total, 8 up, 8 in 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:14.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:14.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:13 smithi174 ceph-mon[39523]: pgmap v297: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi174-79693-36"}]': finished 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-27"}]': finished 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: osdmap e231: 8 total, 8 up, 8 in 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:13 smithi184 ceph-mon[33513]: pgmap v297: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T06:49:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[29878]: osdmap e232: 8 total, 8 up, 8 in 2024-02-10T06:49:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi174-79693-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:15.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[39523]: osdmap e232: 8 total, 8 up, 8 in 2024-02-10T06:49:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:15.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi174-79693-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:15.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:15.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:15 smithi184 ceph-mon[33513]: osdmap e232: 8 total, 8 up, 8 in 2024-02-10T06:49:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi174-79693-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: pgmap v300: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: osdmap e233: 8 total, 8 up, 8 in 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi174-79693-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: osdmap e234: 8 total, 8 up, 8 in 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:16.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: pgmap v300: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: osdmap e233: 8 total, 8 up, 8 in 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi174-79693-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: osdmap e234: 8 total, 8 up, 8 in 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:16.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/19956105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: pgmap v300: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: osdmap e233: 8 total, 8 up, 8 in 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]: dispatch 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi174-79693-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-46"}]': finished 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: osdmap e234: 8 total, 8 up, 8 in 2024-02-10T06:49:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[29878]: osdmap e235: 8 total, 8 up, 8 in 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[39523]: osdmap e235: 8 total, 8 up, 8 in 2024-02-10T06:49:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi174-79705-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:17 smithi184 ceph-mon[33513]: osdmap e235: 8 total, 8 up, 8 in 2024-02-10T06:49:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:18.161 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:49:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:49:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:49:18.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:18 smithi174 ceph-mon[29878]: pgmap v303: 236 pgs: 8 creating+peering, 12 creating+activating, 216 active+clean; 4.6 MiB data, 568 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:49:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:18 smithi174 ceph-mon[29878]: osdmap e236: 8 total, 8 up, 8 in 2024-02-10T06:49:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:18 smithi174 ceph-mon[39523]: pgmap v303: 236 pgs: 8 creating+peering, 12 creating+activating, 216 active+clean; 4.6 MiB data, 568 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:49:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:18 smithi174 ceph-mon[39523]: osdmap e236: 8 total, 8 up, 8 in 2024-02-10T06:49:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:18 smithi184 ceph-mon[33513]: pgmap v303: 236 pgs: 8 creating+peering, 12 creating+activating, 216 active+clean; 4.6 MiB data, 568 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:49:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:18 smithi184 ceph-mon[33513]: osdmap e236: 8 total, 8 up, 8 in 2024-02-10T06:49:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: osdmap e237: 8 total, 8 up, 8 in 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: osdmap e237: 8 total, 8 up, 8 in 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:19.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi174-79705-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: osdmap e237: 8 total, 8 up, 8 in 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]: dispatch 2024-02-10T06:49:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: pgmap v306: 236 pgs: 8 unknown, 12 creating+activating, 216 active+clean; 4.6 MiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]': finished 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: osdmap e238: 8 total, 8 up, 8 in 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]: dispatch 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]: dispatch 2024-02-10T06:49:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: pgmap v306: 236 pgs: 8 unknown, 12 creating+activating, 216 active+clean; 4.6 MiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]': finished 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: osdmap e238: 8 total, 8 up, 8 in 2024-02-10T06:49:20.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:20.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: pgmap v306: 236 pgs: 8 unknown, 12 creating+activating, 216 active+clean; 4.6 MiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/466082366' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi174-79693-37"}]': finished 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-29"}]': finished 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: osdmap e238: 8 total, 8 up, 8 in 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]: dispatch 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]: dispatch 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:20.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]': finished 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: osdmap e239: 8 total, 8 up, 8 in 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:21.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]': finished 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: osdmap e239: 8 total, 8 up, 8 in 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:21.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:21.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-29", "mode": "writeback"}]': finished 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi174-79693-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: osdmap e239: 8 total, 8 up, 8 in 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:23.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:22 smithi184 ceph-mon[33513]: pgmap v309: 228 pgs: 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T06:49:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:22 smithi184 ceph-mon[33513]: osdmap e240: 8 total, 8 up, 8 in 2024-02-10T06:49:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[29878]: pgmap v309: 228 pgs: 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T06:49:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[29878]: osdmap e240: 8 total, 8 up, 8 in 2024-02-10T06:49:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[39523]: pgmap v309: 228 pgs: 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T06:49:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2842164700' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[39523]: osdmap e240: 8 total, 8 up, 8 in 2024-02-10T06:49:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]: dispatch 2024-02-10T06:49:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:23 smithi184 ceph-mon[33513]: osdmap e241: 8 total, 8 up, 8 in 2024-02-10T06:49:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:24.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:24.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi174-79705-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[29878]: osdmap e241: 8 total, 8 up, 8 in 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi174-79705-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi174-79693-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi174-79705-47"}]': finished 2024-02-10T06:49:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[39523]: osdmap e241: 8 total, 8 up, 8 in 2024-02-10T06:49:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:24.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi174-79705-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:24 smithi184 ceph-mon[33513]: pgmap v312: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T06:49:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi174-79705-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:24 smithi184 ceph-mon[33513]: osdmap e242: 8 total, 8 up, 8 in 2024-02-10T06:49:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi174-79705-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:25.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:25.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[29878]: pgmap v312: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi174-79705-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[29878]: osdmap e242: 8 total, 8 up, 8 in 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi174-79705-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[39523]: pgmap v312: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi174-79705-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[39523]: osdmap e242: 8 total, 8 up, 8 in 2024-02-10T06:49:25.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi174-79705-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:25.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:25.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:26.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:26.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:26.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:25 smithi184 ceph-mon[33513]: osdmap e243: 8 total, 8 up, 8 in 2024-02-10T06:49:26.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:26.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:26.117 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[29878]: osdmap e243: 8 total, 8 up, 8 in 2024-02-10T06:49:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[39523]: osdmap e243: 8 total, 8 up, 8 in 2024-02-10T06:49:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]: dispatch 2024-02-10T06:49:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:26.375 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: pgmap v315: 228 pgs: 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi174-79705-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]': finished 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: osdmap e244: 8 total, 8 up, 8 in 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: pgmap v315: 228 pgs: 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi174-79705-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]': finished 2024-02-10T06:49:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: osdmap e244: 8 total, 8 up, 8 in 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: pgmap v315: 228 pgs: 228 active+clean; 8.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi174-79705-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-29"}]': finished 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: osdmap e244: 8 total, 8 up, 8 in 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3011356224' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: osdmap e245: 8 total, 8 up, 8 in 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:28.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:49:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: osdmap e245: 8 total, 8 up, 8 in 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi174-79693-38"}]': finished 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: osdmap e245: 8 total, 8 up, 8 in 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:28.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: pgmap v318: 204 pgs: 8 creating+peering, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: osdmap e246: 8 total, 8 up, 8 in 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:29.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: pgmap v318: 204 pgs: 8 creating+peering, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: osdmap e246: 8 total, 8 up, 8 in 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: pgmap v318: 204 pgs: 8 creating+peering, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi174-79693-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: osdmap e246: 8 total, 8 up, 8 in 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: pgmap v321: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: osdmap e247: 8 total, 8 up, 8 in 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: osdmap e248: 8 total, 8 up, 8 in 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: pgmap v321: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: osdmap e247: 8 total, 8 up, 8 in 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:30.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: osdmap e248: 8 total, 8 up, 8 in 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:30.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: pgmap v321: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: osdmap e247: 8 total, 8 up, 8 in 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]: dispatch 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi174-79693-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2839051812' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi174-79705-48"}]': finished 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: osdmap e248: 8 total, 8 up, 8 in 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]': finished 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: osdmap e249: 8 total, 8 up, 8 in 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]: dispatch 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]: dispatch 2024-02-10T06:49:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]': finished 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: osdmap e249: 8 total, 8 up, 8 in 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]: dispatch 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]: dispatch 2024-02-10T06:49:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-31"}]': finished 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi174-79705-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: osdmap e249: 8 total, 8 up, 8 in 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]: dispatch 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]: dispatch 2024-02-10T06:49:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:32.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:49:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:49:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]': finished 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]': finished 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:49:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:49:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-31", "mode": "writeback"}]': finished 2024-02-10T06:49:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: pgmap v324: 236 pgs: 6 creating+activating, 230 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: osdmap e250: 8 total, 8 up, 8 in 2024-02-10T06:49:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: osdmap e251: 8 total, 8 up, 8 in 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:33.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: pgmap v324: 236 pgs: 6 creating+activating, 230 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: osdmap e250: 8 total, 8 up, 8 in 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: osdmap e251: 8 total, 8 up, 8 in 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: pgmap v324: 236 pgs: 6 creating+activating, 230 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: osdmap e250: 8 total, 8 up, 8 in 2024-02-10T06:49:33.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi174-79705-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: osdmap e251: 8 total, 8 up, 8 in 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2228402183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]: dispatch 2024-02-10T06:49:33.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]: dispatch 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: pgmap v327: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]': finished 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: osdmap e252: 8 total, 8 up, 8 in 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:34.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: pgmap v327: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]': finished 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: osdmap e252: 8 total, 8 up, 8 in 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:34.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: pgmap v327: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-31"}]': finished 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi174-79693-39"}]': finished 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: osdmap e252: 8 total, 8 up, 8 in 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:34.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: osdmap e253: 8 total, 8 up, 8 in 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:36.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: osdmap e253: 8 total, 8 up, 8 in 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi174-79693-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: osdmap e253: 8 total, 8 up, 8 in 2024-02-10T06:49:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:36.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:49:37.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: pgmap v330: 196 pgs: 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: osdmap e254: 8 total, 8 up, 8 in 2024-02-10T06:49:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: pgmap v330: 196 pgs: 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: osdmap e254: 8 total, 8 up, 8 in 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: pgmap v330: 196 pgs: 196 active+clean; 4.6 MiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2237534024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: osdmap e254: 8 total, 8 up, 8 in 2024-02-10T06:49:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]: dispatch 2024-02-10T06:49:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: osdmap e255: 8 total, 8 up, 8 in 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:38.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:49:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: osdmap e255: 8 total, 8 up, 8 in 2024-02-10T06:49:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi174-79693-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi174-79705-49"}]': finished 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: osdmap e255: 8 total, 8 up, 8 in 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:38.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: pgmap v333: 236 pgs: 25 creating+peering, 15 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: osdmap e256: 8 total, 8 up, 8 in 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: pgmap v333: 236 pgs: 25 creating+peering, 15 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: osdmap e256: 8 total, 8 up, 8 in 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: pgmap v333: 236 pgs: 25 creating+peering, 15 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi174-79705-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: osdmap e256: 8 total, 8 up, 8 in 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:39.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]': finished 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]: dispatch 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: pgmap v336: 228 pgs: 25 creating+peering, 7 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: osdmap e257: 8 total, 8 up, 8 in 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]: dispatch 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]': finished 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: osdmap e258: 8 total, 8 up, 8 in 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:40.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]': finished 2024-02-10T06:49:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]: dispatch 2024-02-10T06:49:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: pgmap v336: 228 pgs: 25 creating+peering, 7 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: osdmap e257: 8 total, 8 up, 8 in 2024-02-10T06:49:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]: dispatch 2024-02-10T06:49:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]': finished 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: osdmap e258: 8 total, 8 up, 8 in 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-33"}]': finished 2024-02-10T06:49:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: pgmap v336: 228 pgs: 25 creating+peering, 7 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: osdmap e257: 8 total, 8 up, 8 in 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi174-79705-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-33", "mode": "writeback"}]': finished 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: osdmap e258: 8 total, 8 up, 8 in 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2029486788' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:40.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:42.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: osdmap e259: 8 total, 8 up, 8 in 2024-02-10T06:49:42.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:42.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:42.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:42.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:42.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[29878]: pgmap v339: 236 pgs: 1 creating+activating, 235 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T06:49:42.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:42.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:42.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: osdmap e259: 8 total, 8 up, 8 in 2024-02-10T06:49:42.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:42.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:42.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:41 smithi174 ceph-mon[39523]: pgmap v339: 236 pgs: 1 creating+activating, 235 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi174-79693-40"}]': finished 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: osdmap e259: 8 total, 8 up, 8 in 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:41 smithi184 ceph-mon[33513]: pgmap v339: 236 pgs: 1 creating+activating, 235 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]': finished 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: osdmap e260: 8 total, 8 up, 8 in 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]': finished 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: osdmap e260: 8 total, 8 up, 8 in 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:43.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-33"}]': finished 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi174-79693-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: osdmap e260: 8 total, 8 up, 8 in 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[29878]: pgmap v342: 196 pgs: 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[29878]: osdmap e261: 8 total, 8 up, 8 in 2024-02-10T06:49:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[39523]: pgmap v342: 196 pgs: 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[39523]: osdmap e261: 8 total, 8 up, 8 in 2024-02-10T06:49:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3744660609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:43 smithi184 ceph-mon[33513]: pgmap v342: 196 pgs: 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:49:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:43 smithi184 ceph-mon[33513]: osdmap e261: 8 total, 8 up, 8 in 2024-02-10T06:49:44.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]: dispatch 2024-02-10T06:49:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[29878]: osdmap e262: 8 total, 8 up, 8 in 2024-02-10T06:49:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[39523]: osdmap e262: 8 total, 8 up, 8 in 2024-02-10T06:49:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:45.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:44 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi174-79693-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi174-79705-50"}]': finished 2024-02-10T06:49:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:44 smithi184 ceph-mon[33513]: osdmap e262: 8 total, 8 up, 8 in 2024-02-10T06:49:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: osdmap e263: 8 total, 8 up, 8 in 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1049640564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[29878]: pgmap v345: 268 pgs: 72 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: osdmap e263: 8 total, 8 up, 8 in 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1049640564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:45 smithi174 ceph-mon[39523]: pgmap v345: 268 pgs: 72 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:46.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:49:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: osdmap e263: 8 total, 8 up, 8 in 2024-02-10T06:49:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1049640564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:46.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:45 smithi184 ceph-mon[33513]: pgmap v345: 268 pgs: 72 unknown, 196 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: osdmap e264: 8 total, 8 up, 8 in 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: osdmap e264: 8 total, 8 up, 8 in 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi174-79705-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: osdmap e264: 8 total, 8 up, 8 in 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:49:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:49:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: osdmap e265: 8 total, 8 up, 8 in 2024-02-10T06:49:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: pgmap v348: 228 pgs: 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]': finished 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:48.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: osdmap e266: 8 total, 8 up, 8 in 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: osdmap e265: 8 total, 8 up, 8 in 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: pgmap v348: 228 pgs: 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]': finished 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: osdmap e266: 8 total, 8 up, 8 in 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: osdmap e265: 8 total, 8 up, 8 in 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]: dispatch 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1233724491' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]: dispatch 2024-02-10T06:49:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: pgmap v348: 228 pgs: 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-35", "mode": "writeback"}]': finished 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi174-79693-41"}]': finished 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: osdmap e266: 8 total, 8 up, 8 in 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:48.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:49.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:49.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:49.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi174-79693-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[29878]: pgmap v351: 228 pgs: 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[29878]: osdmap e267: 8 total, 8 up, 8 in 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[39523]: pgmap v351: 228 pgs: 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[39523]: osdmap e267: 8 total, 8 up, 8 in 2024-02-10T06:49:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:50 smithi184 ceph-mon[33513]: pgmap v351: 228 pgs: 228 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:50 smithi184 ceph-mon[33513]: osdmap e267: 8 total, 8 up, 8 in 2024-02-10T06:49:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi174-79705-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: osdmap e268: 8 total, 8 up, 8 in 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: osdmap e269: 8 total, 8 up, 8 in 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: osdmap e268: 8 total, 8 up, 8 in 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: osdmap e269: 8 total, 8 up, 8 in 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:51.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: osdmap e268: 8 total, 8 up, 8 in 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi174-79693-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: osdmap e269: 8 total, 8 up, 8 in 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[29878]: pgmap v354: 244 pgs: 7 creating+peering, 5 creating+activating, 1 unknown, 231 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:49:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[29878]: osdmap e270: 8 total, 8 up, 8 in 2024-02-10T06:49:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[39523]: pgmap v354: 244 pgs: 7 creating+peering, 5 creating+activating, 1 unknown, 231 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:49:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[39523]: osdmap e270: 8 total, 8 up, 8 in 2024-02-10T06:49:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:52 smithi184 ceph-mon[33513]: pgmap v354: 244 pgs: 7 creating+peering, 5 creating+activating, 1 unknown, 231 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:49:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:52 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:52 smithi184 ceph-mon[33513]: osdmap e270: 8 total, 8 up, 8 in 2024-02-10T06:49:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: osdmap e271: 8 total, 8 up, 8 in 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: pgmap v357: 228 pgs: 228 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: osdmap e271: 8 total, 8 up, 8 in 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: pgmap v357: 228 pgs: 228 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: osdmap e271: 8 total, 8 up, 8 in 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/472820745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]: dispatch 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: pgmap v357: 228 pgs: 228 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:49:54.963 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (6659 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7011 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7030 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6984 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7263 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (6818 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7045 ms) 2024-02-10T06:49:54.964 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7042 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6854 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7044 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6933 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7045 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6980 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7035 ms) 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-10T06:49:54.965 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7036 ms) 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7035 ms) 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111815 ms total) 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [----------] Global test environment tear-down 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (208160 ms total) 2024-02-10T06:49:54.966 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio: [ PASSED ] 42 tests. 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: osdmap e272: 8 total, 8 up, 8 in 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: osdmap e272: 8 total, 8 up, 8 in 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:55.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi174-79705-52"}]': finished 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: osdmap e272: 8 total, 8 up, 8 in 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2583503610' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]: dispatch 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]: dispatch 2024-02-10T06:49:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:55.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:55.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:49:56.441 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: osdmap e273: 8 total, 8 up, 8 in 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi174-79705-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[29878]: pgmap v360: 228 pgs: 228 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: osdmap e273: 8 total, 8 up, 8 in 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi174-79705-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:56 smithi174 ceph-mon[39523]: pgmap v360: 228 pgs: 228 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-35"}]': finished 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi174-79693-42"}]': finished 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: osdmap e273: 8 total, 8 up, 8 in 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi174-79705-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:49:56.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:56 smithi184 ceph-mon[33513]: pgmap v360: 228 pgs: 228 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[29878]: osdmap e274: 8 total, 8 up, 8 in 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi174-79705-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[29878]: osdmap e275: 8 total, 8 up, 8 in 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[39523]: osdmap e274: 8 total, 8 up, 8 in 2024-02-10T06:49:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi174-79705-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:49:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[39523]: osdmap e275: 8 total, 8 up, 8 in 2024-02-10T06:49:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:57.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:57.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:49:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:57 smithi184 ceph-mon[33513]: osdmap e274: 8 total, 8 up, 8 in 2024-02-10T06:49:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi174-79705-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:49:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:57 smithi184 ceph-mon[33513]: osdmap e275: 8 total, 8 up, 8 in 2024-02-10T06:49:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:49:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:49:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:49:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[39523]: pgmap v363: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[39523]: osdmap e276: 8 total, 8 up, 8 in 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[29878]: pgmap v363: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:58.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:58.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:58.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[29878]: osdmap e276: 8 total, 8 up, 8 in 2024-02-10T06:49:58.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:58.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:58 smithi184 ceph-mon[33513]: pgmap v363: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:49:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:58 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:49:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:49:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:58 smithi184 ceph-mon[33513]: osdmap e276: 8 total, 8 up, 8 in 2024-02-10T06:49:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:49:58.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: osdmap e277: 8 total, 8 up, 8 in 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: pgmap v366: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:50:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]': finished 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: osdmap e278: 8 total, 8 up, 8 in 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]: dispatch 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]: dispatch 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: osdmap e277: 8 total, 8 up, 8 in 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: pgmap v366: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:50:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]': finished 2024-02-10T06:50:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: osdmap e278: 8 total, 8 up, 8 in 2024-02-10T06:50:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]: dispatch 2024-02-10T06:50:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:50:00.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:49:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]: dispatch 2024-02-10T06:50:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: osdmap e277: 8 total, 8 up, 8 in 2024-02-10T06:50:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:50:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: pgmap v366: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:50:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-37"}]': finished 2024-02-10T06:50:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: osdmap e278: 8 total, 8 up, 8 in 2024-02-10T06:50:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]: dispatch 2024-02-10T06:50:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]: dispatch 2024-02-10T06:50:00.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:49:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]: dispatch 2024-02-10T06:50:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-10T06:50:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T06:50:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T06:50:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T06:50:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]': finished 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: osdmap e279: 8 total, 8 up, 8 in 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:01.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]': finished 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: osdmap e279: 8 total, 8 up, 8 in 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:01.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:01.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:01.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2974871106' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi174-79705-53"}]': finished 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-37", "mode": "writeback"}]': finished 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: osdmap e279: 8 total, 8 up, 8 in 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: pgmap v369: 228 pgs: 228 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:50:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:50:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: osdmap e280: 8 total, 8 up, 8 in 2024-02-10T06:50:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:03.125 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:03.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:03.126 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:03.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: pgmap v369: 228 pgs: 228 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:50:03.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:03.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:50:03.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: osdmap e280: 8 total, 8 up, 8 in 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: pgmap v369: 228 pgs: 228 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi174-79705-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: osdmap e280: 8 total, 8 up, 8 in 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:03.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]: dispatch 2024-02-10T06:50:04.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:03 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:04.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]': finished 2024-02-10T06:50:04.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:03 smithi184 ceph-mon[33513]: osdmap e281: 8 total, 8 up, 8 in 2024-02-10T06:50:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:03 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]': finished 2024-02-10T06:50:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:03 smithi174 ceph-mon[29878]: osdmap e281: 8 total, 8 up, 8 in 2024-02-10T06:50:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:03 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-37"}]': finished 2024-02-10T06:50:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:03 smithi174 ceph-mon[39523]: osdmap e281: 8 total, 8 up, 8 in 2024-02-10T06:50:05.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:04 smithi184 ceph-mon[33513]: pgmap v372: 228 pgs: 228 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:50:05.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:05.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:04 smithi184 ceph-mon[33513]: osdmap e282: 8 total, 8 up, 8 in 2024-02-10T06:50:05.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:04 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[29878]: pgmap v372: 228 pgs: 228 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[29878]: osdmap e282: 8 total, 8 up, 8 in 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[39523]: pgmap v372: 228 pgs: 228 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi174-79705-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[39523]: osdmap e282: 8 total, 8 up, 8 in 2024-02-10T06:50:05.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:04 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: osdmap e283: 8 total, 8 up, 8 in 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: osdmap e284: 8 total, 8 up, 8 in 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:06.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:06.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:06.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: osdmap e283: 8 total, 8 up, 8 in 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: osdmap e284: 8 total, 8 up, 8 in 2024-02-10T06:50:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: osdmap e283: 8 total, 8 up, 8 in 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:06.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: osdmap e284: 8 total, 8 up, 8 in 2024-02-10T06:50:06.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:06.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:06.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:06.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:06.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: pgmap v375: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: osdmap e285: 8 total, 8 up, 8 in 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:07.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: pgmap v375: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: osdmap e285: 8 total, 8 up, 8 in 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: pgmap v375: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: osdmap e285: 8 total, 8 up, 8 in 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3193203174' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:07.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]: dispatch 2024-02-10T06:50:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:50:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:50:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: pgmap v378: 228 pgs: 228 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]': finished 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: osdmap e286: 8 total, 8 up, 8 in 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]: dispatch 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]: dispatch 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:09.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi174-79705-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: pgmap v378: 228 pgs: 228 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]': finished 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: osdmap e286: 8 total, 8 up, 8 in 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]: dispatch 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]: dispatch 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi174-79705-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: pgmap v378: 228 pgs: 228 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-39"}]': finished 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi174-79705-54"}]': finished 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: osdmap e286: 8 total, 8 up, 8 in 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]: dispatch 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]: dispatch 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:09.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi174-79705-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]': finished 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi174-79705-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: osdmap e287: 8 total, 8 up, 8 in 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi174-79705-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]': finished 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi174-79705-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: osdmap e287: 8 total, 8 up, 8 in 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi174-79705-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-39", "mode": "writeback"}]': finished 2024-02-10T06:50:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi174-79705-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: osdmap e287: 8 total, 8 up, 8 in 2024-02-10T06:50:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi174-79705-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:50:11.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: pgmap v381: 228 pgs: 228 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: osdmap e288: 8 total, 8 up, 8 in 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi174-79705-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]': finished 2024-02-10T06:50:11.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:11 smithi184 ceph-mon[33513]: osdmap e289: 8 total, 8 up, 8 in 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: pgmap v381: 228 pgs: 228 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: osdmap e288: 8 total, 8 up, 8 in 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi174-79705-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]': finished 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[39523]: osdmap e289: 8 total, 8 up, 8 in 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: pgmap v381: 228 pgs: 228 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: osdmap e288: 8 total, 8 up, 8 in 2024-02-10T06:50:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]: dispatch 2024-02-10T06:50:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi174-79705-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-39"}]': finished 2024-02-10T06:50:11.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:11 smithi174 ceph-mon[29878]: osdmap e289: 8 total, 8 up, 8 in 2024-02-10T06:50:11.875 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-02-10T06:50:11.875 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-10T06:50:11.875 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-10T06:50:11.875 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: seed 79953 2024-02-10T06:50:11.875 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (296 ms) 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (16864 ms) 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (37 ms) 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (17197 ms total) 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2024-02-10T06:50:11.876 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7845 ms) 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (9107 ms) 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10003 ms) 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [3] 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [4,3] 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [5,4,3] 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting some heads 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-10T06:50:11.877 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: promoting from clones for snap 3 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: waiting for scrubs... 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: done waiting 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46010 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10033 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7983 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8025 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8062 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10074 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8852 ms) 2024-02-10T06:50:11.878 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10075 ms) 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12965 ms) 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7976 ms) 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8285 ms) 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13063 ms) 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7658 ms) 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-10T06:50:11.879 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8019 ms) 2024-02-10T06:50:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:12 smithi174 ceph-mon[29878]: pgmap v384: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 107 op/s 2024-02-10T06:50:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:12 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:12 smithi174 ceph-mon[29878]: osdmap e290: 8 total, 8 up, 8 in 2024-02-10T06:50:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:12 smithi174 ceph-mon[39523]: pgmap v384: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 107 op/s 2024-02-10T06:50:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:12 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:12 smithi174 ceph-mon[39523]: osdmap e290: 8 total, 8 up, 8 in 2024-02-10T06:50:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:12 smithi184 ceph-mon[33513]: pgmap v384: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 107 op/s 2024-02-10T06:50:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:12 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:12 smithi184 ceph-mon[33513]: osdmap e290: 8 total, 8 up, 8 in 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[29878]: osdmap e291: 8 total, 8 up, 8 in 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[39523]: osdmap e291: 8 total, 8 up, 8 in 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:13 smithi184 ceph-mon[33513]: osdmap e291: 8 total, 8 up, 8 in 2024-02-10T06:50:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: pgmap v387: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: osdmap e292: 8 total, 8 up, 8 in 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: pgmap v387: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T06:50:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: osdmap e292: 8 total, 8 up, 8 in 2024-02-10T06:50:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:15.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: pgmap v387: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: osdmap e292: 8 total, 8 up, 8 in 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]: dispatch 2024-02-10T06:50:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: osdmap e293: 8 total, 8 up, 8 in 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:16.162 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi174-79705-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:16.163 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:16.163 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: osdmap e293: 8 total, 8 up, 8 in 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi174-79705-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2430533319' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi174-79705-55"}]': finished 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: osdmap e293: 8 total, 8 up, 8 in 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi174-79705-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: pgmap v390: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:16.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:50:16.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:50:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi174-79705-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: osdmap e294: 8 total, 8 up, 8 in 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi174-79705-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi174-79705-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: osdmap e294: 8 total, 8 up, 8 in 2024-02-10T06:50:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi174-79705-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:17.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi174-79705-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: osdmap e294: 8 total, 8 up, 8 in 2024-02-10T06:50:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi174-79705-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:18.127 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:50:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:50:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: osdmap e295: 8 total, 8 up, 8 in 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi174-79705-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:50:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[29878]: osdmap e296: 8 total, 8 up, 8 in 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: osdmap e295: 8 total, 8 up, 8 in 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi174-79705-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:50:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:18 smithi174 ceph-mon[39523]: osdmap e296: 8 total, 8 up, 8 in 2024-02-10T06:50:18.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:50:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: pgmap v393: 228 pgs: 228 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: osdmap e295: 8 total, 8 up, 8 in 2024-02-10T06:50:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:50:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi174-79705-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:50:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:18 smithi184 ceph-mon[33513]: osdmap e296: 8 total, 8 up, 8 in 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: osdmap e297: 8 total, 8 up, 8 in 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]: dispatch 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]: dispatch 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: pgmap v396: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: osdmap e297: 8 total, 8 up, 8 in 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]: dispatch 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]: dispatch 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: pgmap v396: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:20.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:19 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: osdmap e297: 8 total, 8 up, 8 in 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]: dispatch 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]: dispatch 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: pgmap v396: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:19 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]': finished 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[29878]: osdmap e298: 8 total, 8 up, 8 in 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]': finished 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[39523]: osdmap e298: 8 total, 8 up, 8 in 2024-02-10T06:50:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:20 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-41"}]': finished 2024-02-10T06:50:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:20 smithi184 ceph-mon[33513]: osdmap e298: 8 total, 8 up, 8 in 2024-02-10T06:50:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:20 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[29878]: osdmap e299: 8 total, 8 up, 8 in 2024-02-10T06:50:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[29878]: pgmap v399: 196 pgs: 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[39523]: osdmap e299: 8 total, 8 up, 8 in 2024-02-10T06:50:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[39523]: pgmap v399: 196 pgs: 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:21 smithi184 ceph-mon[33513]: osdmap e299: 8 total, 8 up, 8 in 2024-02-10T06:50:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]: dispatch 2024-02-10T06:50:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:21 smithi184 ceph-mon[33513]: pgmap v399: 196 pgs: 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3611983734' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi174-79705-56"}]': finished 2024-02-10T06:50:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: osdmap e300: 8 total, 8 up, 8 in 2024-02-10T06:50:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi174-79705-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi174-79705-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: osdmap e301: 8 total, 8 up, 8 in 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi174-79705-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi174-79953-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: osdmap e300: 8 total, 8 up, 8 in 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi174-79705-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi174-79705-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: osdmap e301: 8 total, 8 up, 8 in 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi174-79705-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi174-79953-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:23.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: osdmap e300: 8 total, 8 up, 8 in 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi174-79705-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi174-79705-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: osdmap e301: 8 total, 8 up, 8 in 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi174-79705-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi174-79953-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:24 smithi174 ceph-mon[29878]: pgmap v402: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:24 smithi174 ceph-mon[39523]: pgmap v402: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:24.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:24 smithi184 ceph-mon[33513]: pgmap v402: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: osdmap e302: 8 total, 8 up, 8 in 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi174-79705-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]': finished 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: osdmap e303: 8 total, 8 up, 8 in 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: osdmap e302: 8 total, 8 up, 8 in 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi174-79705-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]': finished 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: osdmap e303: 8 total, 8 up, 8 in 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: osdmap e302: 8 total, 8 up, 8 in 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi174-79705-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_count","val": "8"}]': finished 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: osdmap e303: 8 total, 8 up, 8 in 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[29878]: pgmap v405: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[29878]: osdmap e304: 8 total, 8 up, 8 in 2024-02-10T06:50:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T06:50:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T06:50:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[39523]: pgmap v405: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[39523]: osdmap e304: 8 total, 8 up, 8 in 2024-02-10T06:50:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T06:50:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T06:50:26.624 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:50:26.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-mon[33513]: pgmap v405: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:26.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:26.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:26.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-mon[33513]: osdmap e304: 8 total, 8 up, 8 in 2024-02-10T06:50:26.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T06:50:26.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T06:50:27.148 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: ok, hit_set contains 275:602f83fe:::foo:head 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9027 ms) 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg_num = 32 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 0 ls 1707547827,0 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 1 ls 1707547827,0 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 2 ls 1707547827,0 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 3 ls 1707547827,0 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 4 ls 1707547827,0 2024-02-10T06:50:27.149 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 5 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 6 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 7 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 8 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 9 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 10 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 11 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 12 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 13 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 14 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 15 ls 1707547827,0 2024-02-10T06:50:27.150 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 16 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 17 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 18 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 19 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 20 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 21 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 22 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 23 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 24 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 25 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 26 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 27 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 28 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 29 ls 1707547827,0 2024-02-10T06:50:27.151 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 30 ls 1707547827,0 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg 31 ls 1707547827,0 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: pg_num = 32 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6cac518f:::0:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:02547ec2:::1:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f905c69b:::2:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:cfc208b3:::3:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d83876eb:::4:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b29083e3:::5:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c4fdafeb:::6:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:5c6b0b28:::7:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:bd63b0f1:::8:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e960b815:::9:head 2024-02-10T06:50:27.152 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:52ea6a34:::10:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:89d3ae78:::11:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:de5d7c5f:::12:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:566253c9:::13:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:62a1935d:::14:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:863748b0:::15:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3958e169:::16:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4d4dabf9:::17:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:8391935d:::18:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:28883081:::19:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:69259c59:::20:head 2024-02-10T06:50:27.153 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4bdb80b7:::21:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a11c5d71:::22:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:271af37b:::23:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:95b121be:::24:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:58d1031b:::25:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:0a050783:::26:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c709704c:::27:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:cbe56eaf:::28:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:86b4b162:::29:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:70d89383:::30:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:dd450c7c:::31:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6d5729b1:::32:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c388f3fb:::33:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:56cfea31:::34:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9dbc1bf7:::35:head 2024-02-10T06:50:27.154 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:40b74ccd:::36:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4d5aaf42:::37:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:920f362c:::38:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6cc53222:::39:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9cad833f:::40:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:1ea84d41:::41:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c4480ef6:::42:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a694361e:::43:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d1bd33e9:::44:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ddc2cd5d:::45:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:2b782207:::46:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7b187fca:::47:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:90ecdf6f:::48:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a5ed95fe:::49:head 2024-02-10T06:50:27.155 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ea0eaa55:::50:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f33ef17b:::51:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a0d1b2f6:::52:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:60c5229e:::53:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:edcbc575:::54:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:102cf253:::55:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:efb7fb0b:::56:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:50d0a326:::57:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d4dc5daf:::58:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3a130462:::59:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ec87ed71:::60:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d5bc9454:::61:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3ddfe313:::62:head 2024-02-10T06:50:27.156 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7c2816b9:::63:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:47e00e4d:::64:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c6410c18:::65:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b48ed237:::66:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:cd63ad31:::67:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b179e92b:::68:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:0d9f741a:::69:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6d3352ae:::70:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c6d5c19e:::71:head 2024-02-10T06:50:27.157 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:bc4729c3:::72:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:77e930b9:::73:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:0abeecfd:::74:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b7c37e15:::75:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b6378398:::76:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:02bd68de:::77:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:cc795d2d:::78:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:630d4fea:::79:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e0d29ef5:::80:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:fd6f13d2:::81:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:606461d5:::82:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:eadbdc43:::83:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:8761d0bb:::84:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9ef0186f:::85:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e0d41294:::86:head 2024-02-10T06:50:27.158 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:961de695:::87:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:1423148f:::88:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:633a8fa2:::89:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a8653809:::90:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3dac8b33:::91:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:35aad435:::92:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f6dcc343:::93:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:dbbdad87:::94:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:1cb48ce0:::95:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:03cd461c:::96:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:17a4ea99:::97:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9993c9a7:::98:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6394211c:::99:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:94c7ae57:::100:head 2024-02-10T06:50:27.159 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6fdee5bb:::101:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9a477fd1:::102:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:eb850916:::103:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:affc56b9:::104:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b42dc814:::105:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f319f8f0:::106:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9a40b9de:::107:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:8b524f28:::108:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e3de589f:::109:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:90f90a5b:::110:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a7b4f1d7:::111:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:af51766e:::112:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b6f90bd1:::113:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e0261208:::114:head 2024-02-10T06:50:27.160 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c9569ef7:::115:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:61bebe50:::116:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:fe93412b:::117:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d3d38bee:::118:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3100ba0c:::119:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d0560ada:::120:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f0ea8b35:::121:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:766f231a:::122:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a07a2582:::123:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:bd7c6b3a:::124:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:fb2ddaff:::125:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4408e1fe:::126:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ee1df7a7:::127:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c3002909:::128:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4f48ffa9:::129:head 2024-02-10T06:50:27.161 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:edf38733:::130:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c08425c0:::131:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:5f902d98:::132:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:41ea2c93:::133:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:813cee13:::134:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:0131818d:::135:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:26ba5a85:::136:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:381b8a5a:::137:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:28797e47:::138:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:bfca7f22:::139:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:36807075:::140:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:80b03975:::141:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:5c15709b:::142:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f39ea15e:::143:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ea992956:::144:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:48887b1c:::145:head 2024-02-10T06:50:27.162 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:9f24a9dd:::146:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:987f100b:::147:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d2dd3581:::148:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7fed1808:::149:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c80b70e9:::150:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:85ed90f9:::151:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:36428b24:::152:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d044c34a:::153:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7c18bf58:::154:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d1c21232:::155:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a7a3c575:::156:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:87da0633:::157:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d5ac3822:::158:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3f20522d:::159:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6ca26563:::160:head 2024-02-10T06:50:27.163 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:532ce135:::161:head 2024-02-10T06:50:27.164 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:c78863e6:::162:head 2024-02-10T06:50:27.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:27.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T06:50:27.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:27 smithi184 ceph-mon[33513]: osdmap e305: 8 total, 8 up, 8 in 2024-02-10T06:50:27.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:27.471 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi174-79953-43","var": "pg_num","format": "json"}]: dispatch 2024-02-10T06:50:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T06:50:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[29878]: osdmap e305: 8 total, 8 up, 8 in 2024-02-10T06:50:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:27.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi174-79953-43","var": "pg_num","format": "json"}]: dispatch 2024-02-10T06:50:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T06:50:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[39523]: osdmap e305: 8 total, 8 up, 8 in 2024-02-10T06:50:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi174-79953-43","var": "pg_num","format": "json"}]: dispatch 2024-02-10T06:50:28.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:50:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:50:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: pgmap v408: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]: dispatch 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]: dispatch 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]': finished 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: osdmap e306: 8 total, 8 up, 8 in 2024-02-10T06:50:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: pgmap v408: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]: dispatch 2024-02-10T06:50:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]: dispatch 2024-02-10T06:50:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:28.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]': finished 2024-02-10T06:50:28.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: osdmap e306: 8 total, 8 up, 8 in 2024-02-10T06:50:28.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: pgmap v408: 228 pgs: 228 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]: dispatch 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]: dispatch 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-43"}]': finished 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: osdmap e306: 8 total, 8 up, 8 in 2024-02-10T06:50:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]: dispatch 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: osdmap e307: 8 total, 8 up, 8 in 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: pgmap v411: 196 pgs: 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: osdmap e307: 8 total, 8 up, 8 in 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: pgmap v411: 196 pgs: 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2575744738' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi174-79705-57"}]': finished 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: osdmap e307: 8 total, 8 up, 8 in 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: pgmap v411: 196 pgs: 196 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:30.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[29878]: osdmap e308: 8 total, 8 up, 8 in 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[39523]: osdmap e308: 8 total, 8 up, 8 in 2024-02-10T06:50:31.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi174-79705-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:30 smithi184 ceph-mon[33513]: osdmap e308: 8 total, 8 up, 8 in 2024-02-10T06:50:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[29878]: pgmap v414: 228 pgs: 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[29878]: osdmap e309: 8 total, 8 up, 8 in 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[39523]: pgmap v414: 228 pgs: 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[39523]: osdmap e309: 8 total, 8 up, 8 in 2024-02-10T06:50:32.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:32.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:31 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:50:32.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:31 smithi184 ceph-mon[33513]: pgmap v414: 228 pgs: 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:31 smithi184 ceph-mon[33513]: osdmap e309: 8 total, 8 up, 8 in 2024-02-10T06:50:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:31 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: osdmap e310: 8 total, 8 up, 8 in 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:50:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: osdmap e310: 8 total, 8 up, 8 in 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:50:33.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:32 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi174-79705-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: osdmap e310: 8 total, 8 up, 8 in 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:50:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:32 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]': finished 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[29878]: osdmap e311: 8 total, 8 up, 8 in 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[29878]: pgmap v417: 236 pgs: 8 unknown, 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]': finished 2024-02-10T06:50:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]': finished 2024-02-10T06:50:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[39523]: osdmap e311: 8 total, 8 up, 8 in 2024-02-10T06:50:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:50:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:50:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[39523]: pgmap v417: 236 pgs: 8 unknown, 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:34.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]': finished 2024-02-10T06:50:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_count","val": "3"}]': finished 2024-02-10T06:50:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:33 smithi184 ceph-mon[33513]: osdmap e311: 8 total, 8 up, 8 in 2024-02-10T06:50:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:50:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:50:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:33 smithi184 ceph-mon[33513]: pgmap v417: 236 pgs: 8 unknown, 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_period","val": "3"}]': finished 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: osdmap e312: 8 total, 8 up, 8 in 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: osdmap e313: 8 total, 8 up, 8 in 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: osdmap e312: 8 total, 8 up, 8 in 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:35.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: osdmap e313: 8 total, 8 up, 8 in 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:50:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: osdmap e312: 8 total, 8 up, 8 in 2024-02-10T06:50:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: osdmap e313: 8 total, 8 up, 8 in 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1582593375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]: dispatch 2024-02-10T06:50:35.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[29878]: pgmap v420: 228 pgs: 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[29878]: osdmap e314: 8 total, 8 up, 8 in 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[39523]: pgmap v420: 228 pgs: 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T06:50:36.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:35 smithi174 ceph-mon[39523]: osdmap e314: 8 total, 8 up, 8 in 2024-02-10T06:50:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:35 smithi184 ceph-mon[33513]: pgmap v420: 228 pgs: 7 creating+activating, 23 creating+peering, 198 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi174-79705-58"}]': finished 2024-02-10T06:50:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:35 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T06:50:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:35 smithi184 ceph-mon[33513]: osdmap e314: 8 total, 8 up, 8 in 2024-02-10T06:50:36.274 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:36 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:36 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:38.226 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:50:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:50:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[39523]: osdmap e315: 8 total, 8 up, 8 in 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[39523]: pgmap v423: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[29878]: osdmap e315: 8 total, 8 up, 8 in 2024-02-10T06:50:38.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:38.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:38.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:38 smithi174 ceph-mon[29878]: pgmap v423: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:38 smithi184 ceph-mon[33513]: osdmap e315: 8 total, 8 up, 8 in 2024-02-10T06:50:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:38 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:38 smithi184 ceph-mon[33513]: pgmap v423: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:39 smithi174 ceph-mon[29878]: osdmap e316: 8 total, 8 up, 8 in 2024-02-10T06:50:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:39 smithi174 ceph-mon[29878]: osdmap e317: 8 total, 8 up, 8 in 2024-02-10T06:50:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:39 smithi174 ceph-mon[39523]: osdmap e316: 8 total, 8 up, 8 in 2024-02-10T06:50:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:39 smithi174 ceph-mon[39523]: osdmap e317: 8 total, 8 up, 8 in 2024-02-10T06:50:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:39 smithi184 ceph-mon[33513]: osdmap e316: 8 total, 8 up, 8 in 2024-02-10T06:50:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi174-79705-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:39 smithi184 ceph-mon[33513]: osdmap e317: 8 total, 8 up, 8 in 2024-02-10T06:50:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:40 smithi174 ceph-mon[29878]: pgmap v426: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:40 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:40 smithi174 ceph-mon[29878]: osdmap e318: 8 total, 8 up, 8 in 2024-02-10T06:50:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:40 smithi174 ceph-mon[39523]: pgmap v426: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:40 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:40 smithi174 ceph-mon[39523]: osdmap e318: 8 total, 8 up, 8 in 2024-02-10T06:50:40.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:40 smithi184 ceph-mon[33513]: pgmap v426: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:40 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:40 smithi184 ceph-mon[33513]: osdmap e318: 8 total, 8 up, 8 in 2024-02-10T06:50:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:42 smithi184 ceph-mon[33513]: pgmap v428: 236 pgs: 236 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T06:50:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:42 smithi184 ceph-mon[33513]: osdmap e319: 8 total, 8 up, 8 in 2024-02-10T06:50:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[29878]: pgmap v428: 236 pgs: 236 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[29878]: osdmap e319: 8 total, 8 up, 8 in 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[39523]: pgmap v428: 236 pgs: 236 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[39523]: osdmap e319: 8 total, 8 up, 8 in 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:43 smithi184 ceph-mon[33513]: osdmap e320: 8 total, 8 up, 8 in 2024-02-10T06:50:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:43 smithi184 ceph-mon[33513]: osdmap e321: 8 total, 8 up, 8 in 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[29878]: osdmap e320: 8 total, 8 up, 8 in 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[29878]: osdmap e321: 8 total, 8 up, 8 in 2024-02-10T06:50:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:43.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2993245540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[39523]: osdmap e320: 8 total, 8 up, 8 in 2024-02-10T06:50:43.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]: dispatch 2024-02-10T06:50:43.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi174-79705-59"}]': finished 2024-02-10T06:50:43.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:43 smithi174 ceph-mon[39523]: osdmap e321: 8 total, 8 up, 8 in 2024-02-10T06:50:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:44 smithi184 ceph-mon[33513]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:50:44.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:44 smithi174 ceph-mon[29878]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:50:44.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:44 smithi174 ceph-mon[39523]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:50:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:45 smithi184 ceph-mon[33513]: osdmap e322: 8 total, 8 up, 8 in 2024-02-10T06:50:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1095290463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:45 smithi174 ceph-mon[29878]: osdmap e322: 8 total, 8 up, 8 in 2024-02-10T06:50:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1095290463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:45 smithi174 ceph-mon[39523]: osdmap e322: 8 total, 8 up, 8 in 2024-02-10T06:50:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1095290463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7462ddf6::.RoundTripAppendPP (3021 ms) 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3012 ms) 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3217 ms) 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (2755 ms) 2024-02-10T06:50:46.041 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: setting pool EIO 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: max_success 99, min_failed 100 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4259 ms) 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (2770 ms) 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (129042 ms total) 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3010 ms) 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9087 ms) 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14889 ms) 2024-02-10T06:50:46.042 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3015 ms) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30001 ms total) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3018 ms) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3018 ms total) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13981 ms) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6990 ms) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7038 ms) 2024-02-10T06:50:46.043 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7038 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3009 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7033 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6883 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7021 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7025 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7024 ms) 2024-02-10T06:50:46.044 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-10T06:50:46.045 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7022 ms) 2024-02-10T06:50:46.045 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-10T06:50:46.045 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7027 ms) 2024-02-10T06:50:46.045 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-10T06:50:46.045 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7051 ms) 2024-02-10T06:50:46.045 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-10T06:50:46.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:50:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:50:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:46 smithi184 ceph-mon[33513]: pgmap v434: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:46 smithi184 ceph-mon[33513]: osdmap e323: 8 total, 8 up, 8 in 2024-02-10T06:50:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:46 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:46 smithi184 ceph-mon[33513]: osdmap e324: 8 total, 8 up, 8 in 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[29878]: pgmap v434: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[29878]: osdmap e323: 8 total, 8 up, 8 in 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[29878]: osdmap e324: 8 total, 8 up, 8 in 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[39523]: pgmap v434: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi174-79705-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:46.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[39523]: osdmap e323: 8 total, 8 up, 8 in 2024-02-10T06:50:46.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:46.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:46 smithi174 ceph-mon[39523]: osdmap e324: 8 total, 8 up, 8 in 2024-02-10T06:50:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi174-79705-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi174-79705-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi174-79705-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:48.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:50:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:50:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi174-79705-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: osdmap e325: 8 total, 8 up, 8 in 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi174-79705-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]: dispatch 2024-02-10T06:50:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]: dispatch 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi174-79705-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: osdmap e325: 8 total, 8 up, 8 in 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi174-79705-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]: dispatch 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]: dispatch 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi174-79705-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: osdmap e325: 8 total, 8 up, 8 in 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi174-79705-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]: dispatch 2024-02-10T06:50:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]: dispatch 2024-02-10T06:50:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]': finished 2024-02-10T06:50:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:49 smithi184 ceph-mon[33513]: osdmap e326: 8 total, 8 up, 8 in 2024-02-10T06:50:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi174-79705-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:49 smithi184 ceph-mon[33513]: osdmap e327: 8 total, 8 up, 8 in 2024-02-10T06:50:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]': finished 2024-02-10T06:50:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[29878]: osdmap e326: 8 total, 8 up, 8 in 2024-02-10T06:50:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi174-79705-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:49.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[29878]: osdmap e327: 8 total, 8 up, 8 in 2024-02-10T06:50:49.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-45"}]': finished 2024-02-10T06:50:49.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[39523]: osdmap e326: 8 total, 8 up, 8 in 2024-02-10T06:50:49.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi174-79705-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:49.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:49 smithi174 ceph-mon[39523]: osdmap e327: 8 total, 8 up, 8 in 2024-02-10T06:50:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:50 smithi184 ceph-mon[33513]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T06:50:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:50 smithi184 ceph-mon[33513]: osdmap e328: 8 total, 8 up, 8 in 2024-02-10T06:50:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:51.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:51.124 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[29878]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[29878]: osdmap e328: 8 total, 8 up, 8 in 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[39523]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[39523]: osdmap e328: 8 total, 8 up, 8 in 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:50:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: pgmap v443: 236 pgs: 32 creating+peering, 204 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: osdmap e329: 8 total, 8 up, 8 in 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:52 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: pgmap v443: 236 pgs: 32 creating+peering, 204 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: osdmap e329: 8 total, 8 up, 8 in 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: pgmap v443: 236 pgs: 32 creating+peering, 204 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:50:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: osdmap e329: 8 total, 8 up, 8 in 2024-02-10T06:50:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:50:52.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:52 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: osdmap e330: 8 total, 8 up, 8 in 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]': finished 2024-02-10T06:50:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:53 smithi184 ceph-mon[33513]: osdmap e331: 8 total, 8 up, 8 in 2024-02-10T06:50:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: osdmap e330: 8 total, 8 up, 8 in 2024-02-10T06:50:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:50:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]': finished 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[29878]: osdmap e331: 8 total, 8 up, 8 in 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: osdmap e330: 8 total, 8 up, 8 in 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]: dispatch 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/819921045' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi174-79705-61"}]': finished 2024-02-10T06:50:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-47"}]': finished 2024-02-10T06:50:53.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:53 smithi174 ceph-mon[39523]: osdmap e331: 8 total, 8 up, 8 in 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: pgmap v446: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]: dispatch 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]: dispatch 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:54 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: pgmap v446: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]: dispatch 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]: dispatch 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: pgmap v446: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]: dispatch 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]: dispatch 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:50:54.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:54 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]': finished 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: osdmap e332: 8 total, 8 up, 8 in 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi174-79705-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]': finished 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: osdmap e332: 8 total, 8 up, 8 in 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi174-79705-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-47", "mode": "writeback"}]': finished 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: osdmap e332: 8 total, 8 up, 8 in 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi174-79705-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:50:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:50:56.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: pgmap v449: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: osdmap e333: 8 total, 8 up, 8 in 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi174-79705-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: osdmap e334: 8 total, 8 up, 8 in 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: pgmap v449: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: osdmap e333: 8 total, 8 up, 8 in 2024-02-10T06:50:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi174-79705-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: osdmap e334: 8 total, 8 up, 8 in 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: pgmap v449: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: osdmap e333: 8 total, 8 up, 8 in 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:50:56.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi174-79705-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:50:56.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:50:56.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: osdmap e334: 8 total, 8 up, 8 in 2024-02-10T06:50:56.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:56.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:50:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:57 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:50:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:57 smithi184 ceph-mon[33513]: osdmap e335: 8 total, 8 up, 8 in 2024-02-10T06:50:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:50:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[29878]: osdmap e335: 8 total, 8 up, 8 in 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:50:57.640 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:50:57.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:50:57.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:50:57.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[39523]: osdmap e335: 8 total, 8 up, 8 in 2024-02-10T06:50:57.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:50:57.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:50:57.989 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:50:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:50:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: pgmap v452: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: osdmap e336: 8 total, 8 up, 8 in 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:50:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: pgmap v452: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: osdmap e336: 8 total, 8 up, 8 in 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: pgmap v452: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: osdmap e336: 8 total, 8 up, 8 in 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:50:58.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:50:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:51:00.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: pgmap v455: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:51:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:51:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T06:51:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: osdmap e337: 8 total, 8 up, 8 in 2024-02-10T06:51:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:51:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:51:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: pgmap v455: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: osdmap e337: 8 total, 8 up, 8 in 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:51:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: pgmap v455: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: osdmap e337: 8 total, 8 up, 8 in 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]: dispatch 2024-02-10T06:51:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: osdmap e338: 8 total, 8 up, 8 in 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:01.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: osdmap e338: 8 total, 8 up, 8 in 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3437668887' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi174-79705-62"}]': finished 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: osdmap e338: 8 total, 8 up, 8 in 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:01.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:01.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:01.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: osdmap e339: 8 total, 8 up, 8 in 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: osdmap e339: 8 total, 8 up, 8 in 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi174-79705-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: osdmap e339: 8 total, 8 up, 8 in 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]': finished 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: osdmap e340: 8 total, 8 up, 8 in 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:03 smithi184 ceph-mon[33513]: osdmap e341: 8 total, 8 up, 8 in 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]': finished 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: osdmap e340: 8 total, 8 up, 8 in 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[29878]: osdmap e341: 8 total, 8 up, 8 in 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]': finished 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: osdmap e340: 8 total, 8 up, 8 in 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-47"}]: dispatch 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi174-79705-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:03.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:03 smithi174 ceph-mon[39523]: osdmap e341: 8 total, 8 up, 8 in 2024-02-10T06:51:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[29878]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:51:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[29878]: osdmap e342: 8 total, 8 up, 8 in 2024-02-10T06:51:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:04.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:04.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[39523]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:51:04.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[39523]: osdmap e342: 8 total, 8 up, 8 in 2024-02-10T06:51:04.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:04.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:04.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:04 smithi184 ceph-mon[33513]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:51:04.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:04 smithi184 ceph-mon[33513]: osdmap e342: 8 total, 8 up, 8 in 2024-02-10T06:51:04.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:04.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: osdmap e343: 8 total, 8 up, 8 in 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: osdmap e343: 8 total, 8 up, 8 in 2024-02-10T06:51:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: osdmap e343: 8 total, 8 up, 8 in 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:06.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:51:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: pgmap v464: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:51:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: osdmap e344: 8 total, 8 up, 8 in 2024-02-10T06:51:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:06.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:06.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:06.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: pgmap v464: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:51:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: osdmap e344: 8 total, 8 up, 8 in 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: pgmap v464: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: osdmap e344: 8 total, 8 up, 8 in 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1312624634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:06.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]: dispatch 2024-02-10T06:51:06.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]': finished 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: osdmap e345: 8 total, 8 up, 8 in 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]: dispatch 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]: dispatch 2024-02-10T06:51:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]': finished 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: osdmap e345: 8 total, 8 up, 8 in 2024-02-10T06:51:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:07.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi174-79705-63"}]': finished 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-49"}]': finished 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: osdmap e345: 8 total, 8 up, 8 in 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:51:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:51:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]': finished 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: osdmap e346: 8 total, 8 up, 8 in 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]': finished 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: osdmap e346: 8 total, 8 up, 8 in 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-49", "mode": "readproxy"}]': finished 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi174-79705-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: osdmap e346: 8 total, 8 up, 8 in 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:10 smithi174 ceph-mon[29878]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:51:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:10 smithi174 ceph-mon[29878]: osdmap e347: 8 total, 8 up, 8 in 2024-02-10T06:51:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:10 smithi174 ceph-mon[39523]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:51:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:10 smithi174 ceph-mon[39523]: osdmap e347: 8 total, 8 up, 8 in 2024-02-10T06:51:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:10 smithi184 ceph-mon[33513]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:51:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:10 smithi184 ceph-mon[33513]: osdmap e347: 8 total, 8 up, 8 in 2024-02-10T06:51:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:11 smithi174 ceph-mon[29878]: osdmap e348: 8 total, 8 up, 8 in 2024-02-10T06:51:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:11 smithi174 ceph-mon[39523]: osdmap e348: 8 total, 8 up, 8 in 2024-02-10T06:51:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi174-79705-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:11 smithi184 ceph-mon[33513]: osdmap e348: 8 total, 8 up, 8 in 2024-02-10T06:51:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:12 smithi174 ceph-mon[29878]: pgmap v473: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:51:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:12 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:12 smithi174 ceph-mon[29878]: osdmap e349: 8 total, 8 up, 8 in 2024-02-10T06:51:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:12 smithi174 ceph-mon[39523]: pgmap v473: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:51:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:12 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:12 smithi174 ceph-mon[39523]: osdmap e349: 8 total, 8 up, 8 in 2024-02-10T06:51:12.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:12 smithi184 ceph-mon[33513]: pgmap v473: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:51:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:12 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:12 smithi184 ceph-mon[33513]: osdmap e349: 8 total, 8 up, 8 in 2024-02-10T06:51:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:13 smithi174 ceph-mon[29878]: osdmap e350: 8 total, 8 up, 8 in 2024-02-10T06:51:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:13 smithi174 ceph-mon[39523]: osdmap e350: 8 total, 8 up, 8 in 2024-02-10T06:51:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:13.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:13 smithi184 ceph-mon[33513]: osdmap e350: 8 total, 8 up, 8 in 2024-02-10T06:51:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[29878]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[29878]: osdmap e351: 8 total, 8 up, 8 in 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[39523]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[39523]: osdmap e351: 8 total, 8 up, 8 in 2024-02-10T06:51:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:14.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:14 smithi184 ceph-mon[33513]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T06:51:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:14 smithi184 ceph-mon[33513]: osdmap e351: 8 total, 8 up, 8 in 2024-02-10T06:51:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3223571130' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]: dispatch 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: osdmap e352: 8 total, 8 up, 8 in 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: osdmap e352: 8 total, 8 up, 8 in 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi174-79705-64"}]': finished 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: osdmap e352: 8 total, 8 up, 8 in 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:16.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:51:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:51:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:16 smithi184 ceph-mon[33513]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:16 smithi184 ceph-mon[33513]: osdmap e353: 8 total, 8 up, 8 in 2024-02-10T06:51:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[29878]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[29878]: osdmap e353: 8 total, 8 up, 8 in 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[39523]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi174-79705-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[39523]: osdmap e353: 8 total, 8 up, 8 in 2024-02-10T06:51:16.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[29878]: osdmap e354: 8 total, 8 up, 8 in 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[29878]: osdmap e355: 8 total, 8 up, 8 in 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[39523]: osdmap e354: 8 total, 8 up, 8 in 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-mon[39523]: osdmap e355: 8 total, 8 up, 8 in 2024-02-10T06:51:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:17 smithi184 ceph-mon[33513]: osdmap e354: 8 total, 8 up, 8 in 2024-02-10T06:51:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi174-79705-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:17 smithi184 ceph-mon[33513]: osdmap e355: 8 total, 8 up, 8 in 2024-02-10T06:51:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:51:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:51:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:51:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:18 smithi174 ceph-mon[29878]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:18.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:18 smithi174 ceph-mon[29878]: osdmap e356: 8 total, 8 up, 8 in 2024-02-10T06:51:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:18 smithi174 ceph-mon[39523]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:18 smithi174 ceph-mon[39523]: osdmap e356: 8 total, 8 up, 8 in 2024-02-10T06:51:18.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:18 smithi184 ceph-mon[33513]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:18.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:18 smithi184 ceph-mon[33513]: osdmap e356: 8 total, 8 up, 8 in 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: osdmap e357: 8 total, 8 up, 8 in 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:19.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: osdmap e357: 8 total, 8 up, 8 in 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:19.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: osdmap e357: 8 total, 8 up, 8 in 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:19.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: pgmap v485: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]': finished 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: osdmap e358: 8 total, 8 up, 8 in 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: pgmap v485: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]': finished 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: osdmap e358: 8 total, 8 up, 8 in 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:20.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:20.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: pgmap v485: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:20.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:20.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:51:20.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]': finished 2024-02-10T06:51:20.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:20.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: osdmap e358: 8 total, 8 up, 8 in 2024-02-10T06:51:20.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1546798040' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]: dispatch 2024-02-10T06:51:20.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:20.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:20.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:20.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-49"}]: dispatch 2024-02-10T06:51:21.192 INFO:tasks.workunit.client.0.smithi174.stdout::163:head 2024-02-10T06:51:21.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:5d165639:::164:head 2024-02-10T06:51:21.192 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f43765fc:::165:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b4c720e9:::166:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e694b040:::167:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:afa38db2:::168:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:77ba9f53:::169:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:87495034:::170:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7c96bf0e:::171:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:dbe346cc:::172:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:e943ec24:::173:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f97a9c0c:::174:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:6f26e74d:::175:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4f95e106:::176:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:0e6f2f8f:::177:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:05db05f1:::178:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:38a78d66:::179:head 2024-02-10T06:51:21.193 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:d095610b:::180:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:a1a9d709:::181:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:1e5d39db:::182:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:f7df4fb9:::183:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:03a7f161:::184:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ba70721e:::185:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:28e5662d:::186:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:973d52de:::187:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:4303eb1c:::188:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:b990b48e:::189:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:29b8165b:::190:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:3547f197:::191:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:7e260936:::192:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:1abec7b1:::193:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:10fdda93:::194:head 2024-02-10T06:51:21.194 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:15817eea:::195:head 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:770bab57:::196:head 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:ed9e13e7:::197:head 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:71471a8f:::198:head 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: checking for 277:10fb1d02:::199:head 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8028 ms) 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first is 1707547836 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,1707547841,1707547842,0 2024-02-10T06:51:21.195 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,1707547841,1707547842,0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,1707547841,1707547842,0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,1707547841,1707547842,1707547844,1707547845,0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,1707547841,1707547842,1707547844,1707547845,0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547836,1707547838,1707547839,1707547841,1707547842,1707547844,1707547845,0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707547839,1707547841,1707547842,1707547844,1707547845,1707547847,1707547848,0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first now 1707547839, trimmed 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20139 ms) 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: foo0 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14050 ms) 2024-02-10T06:51:21.196 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-10T06:51:21.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:21.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:21.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:21 smithi174 ceph-mon[29878]: osdmap e359: 8 total, 8 up, 8 in 2024-02-10T06:51:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:21.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:21 smithi174 ceph-mon[39523]: osdmap e359: 8 total, 8 up, 8 in 2024-02-10T06:51:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi174-79705-65"}]': finished 2024-02-10T06:51:21.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:21 smithi184 ceph-mon[33513]: osdmap e359: 8 total, 8 up, 8 in 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: osdmap e360: 8 total, 8 up, 8 in 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: osdmap e360: 8 total, 8 up, 8 in 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi174-79705-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: osdmap e360: 8 total, 8 up, 8 in 2024-02-10T06:51:22.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:22.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:22.874 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[39523]: pgmap v491: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[39523]: osdmap e361: 8 total, 8 up, 8 in 2024-02-10T06:51:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:24.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[29878]: pgmap v491: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:24.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:24.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:24.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[29878]: osdmap e361: 8 total, 8 up, 8 in 2024-02-10T06:51:24.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:24 smithi184 ceph-mon[33513]: pgmap v491: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:51:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:24 smithi184 ceph-mon[33513]: osdmap e361: 8 total, 8 up, 8 in 2024-02-10T06:51:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:51:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:51:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[29878]: osdmap e362: 8 total, 8 up, 8 in 2024-02-10T06:51:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:51:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[39523]: osdmap e362: 8 total, 8 up, 8 in 2024-02-10T06:51:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:25.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi174-79705-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:51:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:25 smithi184 ceph-mon[33513]: osdmap e362: 8 total, 8 up, 8 in 2024-02-10T06:51:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: pgmap v494: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]': finished 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: osdmap e363: 8 total, 8 up, 8 in 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]: dispatch 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]: dispatch 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:51:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]': finished 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[29878]: osdmap e364: 8 total, 8 up, 8 in 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: pgmap v494: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]': finished 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: osdmap e363: 8 total, 8 up, 8 in 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]: dispatch 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]: dispatch 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]': finished 2024-02-10T06:51:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:26 smithi174 ceph-mon[39523]: osdmap e364: 8 total, 8 up, 8 in 2024-02-10T06:51:26.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: pgmap v494: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-51"}]': finished 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: osdmap e363: 8 total, 8 up, 8 in 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]: dispatch 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]: dispatch 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-51", "mode": "writeback"}]': finished 2024-02-10T06:51:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:26 smithi184 ceph-mon[33513]: osdmap e364: 8 total, 8 up, 8 in 2024-02-10T06:51:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:51:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:51:27.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:27.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:27.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:27.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:51:27.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:51:27.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:51:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:51:28.217 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:51:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:51:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:51:28.239 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3031 ms) 2024-02-10T06:51:28.240 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-10T06:51:28.240 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7039 ms) 2024-02-10T06:51:28.240 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-10T06:51:28.240 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7027 ms) 2024-02-10T06:51:28.240 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-10T06:51:28.240 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7017 ms) 2024-02-10T06:51:28.241 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-10T06:51:28.241 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7038 ms) 2024-02-10T06:51:28.241 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-10T06:51:28.241 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7030 ms) 2024-02-10T06:51:28.241 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-10T06:51:28.242 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7042 ms) 2024-02-10T06:51:28.242 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139369 ms total) 2024-02-10T06:51:28.242 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: 2024-02-10T06:51:28.242 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-10T06:51:28.242 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (301430 ms total) 2024-02-10T06:51:28.242 INFO:tasks.workunit.client.0.smithi174.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: osdmap e365: 8 total, 8 up, 8 in 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:28.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: osdmap e365: 8 total, 8 up, 8 in 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:51:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: osdmap e365: 8 total, 8 up, 8 in 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2485688736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]: dispatch 2024-02-10T06:51:28.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:51:28.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[29878]: osdmap e366: 8 total, 8 up, 8 in 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:51:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[39523]: osdmap e366: 8 total, 8 up, 8 in 2024-02-10T06:51:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:51:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:51:29.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi174-79705-66"}]': finished 2024-02-10T06:51:29.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:51:29.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:29 smithi184 ceph-mon[33513]: osdmap e366: 8 total, 8 up, 8 in 2024-02-10T06:51:29.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:51:29.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:51:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:30 smithi184 ceph-mon[33513]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:30 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:51:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:51:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:30 smithi184 ceph-mon[33513]: osdmap e367: 8 total, 8 up, 8 in 2024-02-10T06:51:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:51:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[29878]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[29878]: osdmap e367: 8 total, 8 up, 8 in 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[39523]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:30.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:51:30.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:51:30.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[39523]: osdmap e367: 8 total, 8 up, 8 in 2024-02-10T06:51:30.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:51:30.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:51:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:51:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:31 smithi184 ceph-mon[33513]: osdmap e368: 8 total, 8 up, 8 in 2024-02-10T06:51:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T06:51:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T06:51:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]': finished 2024-02-10T06:51:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:31 smithi184 ceph-mon[33513]: osdmap e369: 8 total, 8 up, 8 in 2024-02-10T06:51:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:51:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[29878]: osdmap e368: 8 total, 8 up, 8 in 2024-02-10T06:51:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T06:51:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T06:51:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]': finished 2024-02-10T06:51:31.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[29878]: osdmap e369: 8 total, 8 up, 8 in 2024-02-10T06:51:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:51:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[39523]: osdmap e368: 8 total, 8 up, 8 in 2024-02-10T06:51:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T06:51:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T06:51:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-51","var": "target_max_objects","val": "1"}]': finished 2024-02-10T06:51:31.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:31 smithi174 ceph-mon[39523]: osdmap e369: 8 total, 8 up, 8 in 2024-02-10T06:51:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:32 smithi184 ceph-mon[33513]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T06:51:32.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:32 smithi174 ceph-mon[29878]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T06:51:32.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:32 smithi174 ceph-mon[39523]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T06:51:33.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:33 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:51:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:33 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:51:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:33 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:51:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:34 smithi184 ceph-mon[33513]: pgmap v505: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T06:51:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:34 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T06:51:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:51:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:51:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[29878]: pgmap v505: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[39523]: pgmap v505: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T06:51:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:51:34.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:51:34.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:36.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:51:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:51:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:36 smithi184 ceph-mon[33513]: pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:37.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:36 smithi174 ceph-mon[29878]: pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:36 smithi174 ceph-mon[39523]: pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:38.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:38.240 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:51:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:51:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:51:39.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:38 smithi184 ceph-mon[33513]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:38 smithi174 ceph-mon[29878]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:38 smithi174 ceph-mon[39523]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:41.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:40 smithi184 ceph-mon[33513]: pgmap v508: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:40 smithi174 ceph-mon[29878]: pgmap v508: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:41.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:40 smithi174 ceph-mon[39523]: pgmap v508: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:51:42.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:42.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:42.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:42.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:43.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:42 smithi184 ceph-mon[33513]: pgmap v509: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:51:43.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:43.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:42 smithi184 ceph-mon[33513]: osdmap e370: 8 total, 8 up, 8 in 2024-02-10T06:51:43.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:43.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[29878]: pgmap v509: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[29878]: osdmap e370: 8 total, 8 up, 8 in 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[39523]: pgmap v509: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[39523]: osdmap e370: 8 total, 8 up, 8 in 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]': finished 2024-02-10T06:51:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[29878]: osdmap e371: 8 total, 8 up, 8 in 2024-02-10T06:51:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]': finished 2024-02-10T06:51:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[39523]: osdmap e371: 8 total, 8 up, 8 in 2024-02-10T06:51:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:44.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:44.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]': finished 2024-02-10T06:51:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:43 smithi184 ceph-mon[33513]: osdmap e371: 8 total, 8 up, 8 in 2024-02-10T06:51:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-51"}]: dispatch 2024-02-10T06:51:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:44 smithi174 ceph-mon[29878]: pgmap v512: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:51:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:44 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T06:51:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:44 smithi174 ceph-mon[29878]: osdmap e372: 8 total, 8 up, 8 in 2024-02-10T06:51:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:44 smithi174 ceph-mon[39523]: pgmap v512: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:51:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:44 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T06:51:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:44 smithi174 ceph-mon[39523]: osdmap e372: 8 total, 8 up, 8 in 2024-02-10T06:51:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:44 smithi184 ceph-mon[33513]: pgmap v512: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:51:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:44 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T06:51:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:44 smithi184 ceph-mon[33513]: osdmap e372: 8 total, 8 up, 8 in 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[39523]: osdmap e373: 8 total, 8 up, 8 in 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[39523]: pgmap v515: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[29878]: osdmap e373: 8 total, 8 up, 8 in 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:45 smithi174 ceph-mon[29878]: pgmap v515: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:46.251 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:45 smithi184 ceph-mon[33513]: osdmap e373: 8 total, 8 up, 8 in 2024-02-10T06:51:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:46.252 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:45 smithi184 ceph-mon[33513]: pgmap v515: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:46.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:51:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: osdmap e374: 8 total, 8 up, 8 in 2024-02-10T06:51:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-53"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-53"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: osdmap e374: 8 total, 8 up, 8 in 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-53"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-53"}]: dispatch 2024-02-10T06:51:47.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: osdmap e374: 8 total, 8 up, 8 in 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-53"}]: dispatch 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-53"}]: dispatch 2024-02-10T06:51:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:47 smithi174 ceph-mon[29878]: osdmap e375: 8 total, 8 up, 8 in 2024-02-10T06:51:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:47 smithi174 ceph-mon[29878]: pgmap v518: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T06:51:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:51:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:51:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:51:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:47 smithi174 ceph-mon[39523]: osdmap e375: 8 total, 8 up, 8 in 2024-02-10T06:51:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:47 smithi174 ceph-mon[39523]: pgmap v518: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T06:51:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:47 smithi184 ceph-mon[33513]: osdmap e375: 8 total, 8 up, 8 in 2024-02-10T06:51:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:47 smithi184 ceph-mon[33513]: pgmap v518: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T06:51:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:48 smithi174 ceph-mon[29878]: osdmap e376: 8 total, 8 up, 8 in 2024-02-10T06:51:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:48 smithi174 ceph-mon[39523]: osdmap e376: 8 total, 8 up, 8 in 2024-02-10T06:51:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:49.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:48 smithi184 ceph-mon[33513]: osdmap e376: 8 total, 8 up, 8 in 2024-02-10T06:51:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: osdmap e377: 8 total, 8 up, 8 in 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-55"}]: dispatch 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-55"}]: dispatch 2024-02-10T06:51:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[29878]: pgmap v521: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: osdmap e377: 8 total, 8 up, 8 in 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-55"}]: dispatch 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-55"}]: dispatch 2024-02-10T06:51:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:49 smithi174 ceph-mon[39523]: pgmap v521: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: osdmap e377: 8 total, 8 up, 8 in 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-55"}]: dispatch 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-55"}]: dispatch 2024-02-10T06:51:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:49 smithi184 ceph-mon[33513]: pgmap v521: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:51:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:50 smithi174 ceph-mon[29878]: osdmap e378: 8 total, 8 up, 8 in 2024-02-10T06:51:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:50 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:50 smithi174 ceph-mon[39523]: osdmap e378: 8 total, 8 up, 8 in 2024-02-10T06:51:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:50 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:50 smithi184 ceph-mon[33513]: osdmap e378: 8 total, 8 up, 8 in 2024-02-10T06:51:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:51:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:50 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[29878]: osdmap e379: 8 total, 8 up, 8 in 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[29878]: pgmap v524: 228 pgs: 27 creating+peering, 5 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[39523]: osdmap e379: 8 total, 8 up, 8 in 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:51 smithi174 ceph-mon[39523]: pgmap v524: 228 pgs: 27 creating+peering, 5 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:51 smithi184 ceph-mon[33513]: osdmap e379: 8 total, 8 up, 8 in 2024-02-10T06:51:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:51 smithi184 ceph-mon[33513]: pgmap v524: 228 pgs: 27 creating+peering, 5 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: osdmap e380: 8 total, 8 up, 8 in 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-57"}]: dispatch 2024-02-10T06:51:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-57"}]: dispatch 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: osdmap e380: 8 total, 8 up, 8 in 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-57"}]: dispatch 2024-02-10T06:51:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-57"}]: dispatch 2024-02-10T06:51:53.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: osdmap e380: 8 total, 8 up, 8 in 2024-02-10T06:51:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-57"}]: dispatch 2024-02-10T06:51:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-57"}]: dispatch 2024-02-10T06:51:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:53 smithi174 ceph-mon[29878]: osdmap e381: 8 total, 8 up, 8 in 2024-02-10T06:51:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:53 smithi174 ceph-mon[29878]: pgmap v527: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:53 smithi174 ceph-mon[39523]: osdmap e381: 8 total, 8 up, 8 in 2024-02-10T06:51:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:53 smithi174 ceph-mon[39523]: pgmap v527: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:53 smithi184 ceph-mon[33513]: osdmap e381: 8 total, 8 up, 8 in 2024-02-10T06:51:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:53 smithi184 ceph-mon[33513]: pgmap v527: 196 pgs: 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T06:51:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:54 smithi174 ceph-mon[29878]: osdmap e382: 8 total, 8 up, 8 in 2024-02-10T06:51:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:54 smithi174 ceph-mon[39523]: osdmap e382: 8 total, 8 up, 8 in 2024-02-10T06:51:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:55.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:54 smithi184 ceph-mon[33513]: osdmap e382: 8 total, 8 up, 8 in 2024-02-10T06:51:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: osdmap e383: 8 total, 8 up, 8 in 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: pgmap v530: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-59"}]: dispatch 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-59"}]: dispatch 2024-02-10T06:51:56.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: osdmap e383: 8 total, 8 up, 8 in 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: pgmap v530: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-59"}]: dispatch 2024-02-10T06:51:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-59"}]: dispatch 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: osdmap e383: 8 total, 8 up, 8 in 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: pgmap v530: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-59"}]: dispatch 2024-02-10T06:51:56.255 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-59"}]: dispatch 2024-02-10T06:51:56.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:51:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:51:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:56 smithi174 ceph-mon[29878]: osdmap e384: 8 total, 8 up, 8 in 2024-02-10T06:51:57.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:56 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:56 smithi174 ceph-mon[39523]: osdmap e384: 8 total, 8 up, 8 in 2024-02-10T06:51:57.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:56 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:56 smithi184 ceph-mon[33513]: osdmap e384: 8 total, 8 up, 8 in 2024-02-10T06:51:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:56 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:51:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:51:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:51:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[29878]: osdmap e385: 8 total, 8 up, 8 in 2024-02-10T06:51:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:58.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[29878]: pgmap v533: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:51:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[39523]: osdmap e385: 8 total, 8 up, 8 in 2024-02-10T06:51:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:57 smithi174 ceph-mon[39523]: pgmap v533: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:51:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:57 smithi184 ceph-mon[33513]: osdmap e385: 8 total, 8 up, 8 in 2024-02-10T06:51:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:51:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:57 smithi184 ceph-mon[33513]: pgmap v533: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:51:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:58 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:58 smithi184 ceph-mon[33513]: osdmap e386: 8 total, 8 up, 8 in 2024-02-10T06:51:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:51:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:51:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[29878]: osdmap e386: 8 total, 8 up, 8 in 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:51:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:51:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:51:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[39523]: osdmap e386: 8 total, 8 up, 8 in 2024-02-10T06:51:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:51:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:51:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:51:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: osdmap e387: 8 total, 8 up, 8 in 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: pgmap v536: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-61"}]: dispatch 2024-02-10T06:52:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-61"}]: dispatch 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: osdmap e387: 8 total, 8 up, 8 in 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: pgmap v536: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-61"}]: dispatch 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-61"}]: dispatch 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: osdmap e387: 8 total, 8 up, 8 in 2024-02-10T06:52:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: pgmap v536: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T06:52:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-61"}]: dispatch 2024-02-10T06:52:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-61"}]: dispatch 2024-02-10T06:52:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:01 smithi184 ceph-mon[33513]: osdmap e388: 8 total, 8 up, 8 in 2024-02-10T06:52:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:01 smithi174 ceph-mon[29878]: osdmap e388: 8 total, 8 up, 8 in 2024-02-10T06:52:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:01 smithi174 ceph-mon[39523]: osdmap e388: 8 total, 8 up, 8 in 2024-02-10T06:52:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:02 smithi184 ceph-mon[33513]: pgmap v538: 196 pgs: 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 508 B/s wr, 1 op/s 2024-02-10T06:52:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:02 smithi184 ceph-mon[33513]: osdmap e389: 8 total, 8 up, 8 in 2024-02-10T06:52:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[29878]: pgmap v538: 196 pgs: 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 508 B/s wr, 1 op/s 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[29878]: osdmap e389: 8 total, 8 up, 8 in 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[39523]: pgmap v538: 196 pgs: 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 508 B/s wr, 1 op/s 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[39523]: osdmap e389: 8 total, 8 up, 8 in 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:52:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:03 smithi184 ceph-mon[33513]: osdmap e390: 8 total, 8 up, 8 in 2024-02-10T06:52:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:52:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[29878]: osdmap e390: 8 total, 8 up, 8 in 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[39523]: osdmap e390: 8 total, 8 up, 8 in 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:52:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:04 smithi184 ceph-mon[33513]: pgmap v541: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-02-10T06:52:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:04 smithi184 ceph-mon[33513]: osdmap e391: 8 total, 8 up, 8 in 2024-02-10T06:52:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:04 smithi174 ceph-mon[39523]: pgmap v541: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-02-10T06:52:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:04 smithi174 ceph-mon[39523]: osdmap e391: 8 total, 8 up, 8 in 2024-02-10T06:52:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:04 smithi174 ceph-mon[29878]: pgmap v541: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-02-10T06:52:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:04 smithi174 ceph-mon[29878]: osdmap e391: 8 total, 8 up, 8 in 2024-02-10T06:52:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:05 smithi184 ceph-mon[33513]: osdmap e392: 8 total, 8 up, 8 in 2024-02-10T06:52:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:05 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:05 smithi174 ceph-mon[29878]: osdmap e392: 8 total, 8 up, 8 in 2024-02-10T06:52:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:05 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:05 smithi174 ceph-mon[39523]: osdmap e392: 8 total, 8 up, 8 in 2024-02-10T06:52:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:05 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:06.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:52:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:52:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:06 smithi184 ceph-mon[33513]: pgmap v544: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:06 smithi184 ceph-mon[33513]: osdmap e393: 8 total, 8 up, 8 in 2024-02-10T06:52:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:06 smithi174 ceph-mon[29878]: pgmap v544: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:06 smithi174 ceph-mon[29878]: osdmap e393: 8 total, 8 up, 8 in 2024-02-10T06:52:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:06 smithi174 ceph-mon[39523]: pgmap v544: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:06 smithi174 ceph-mon[39523]: osdmap e393: 8 total, 8 up, 8 in 2024-02-10T06:52:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:07 smithi184 ceph-mon[33513]: osdmap e394: 8 total, 8 up, 8 in 2024-02-10T06:52:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:07 smithi174 ceph-mon[29878]: osdmap e394: 8 total, 8 up, 8 in 2024-02-10T06:52:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:07 smithi174 ceph-mon[39523]: osdmap e394: 8 total, 8 up, 8 in 2024-02-10T06:52:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:08.066 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:52:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:52:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:52:08.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:08 smithi184 ceph-mon[33513]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-10T06:52:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:08 smithi184 ceph-mon[33513]: osdmap e395: 8 total, 8 up, 8 in 2024-02-10T06:52:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:08 smithi174 ceph-mon[29878]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-10T06:52:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:08 smithi174 ceph-mon[29878]: osdmap e395: 8 total, 8 up, 8 in 2024-02-10T06:52:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:08 smithi174 ceph-mon[39523]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-10T06:52:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:08 smithi174 ceph-mon[39523]: osdmap e395: 8 total, 8 up, 8 in 2024-02-10T06:52:11.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:10 smithi184 ceph-mon[33513]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-10T06:52:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:10 smithi174 ceph-mon[29878]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-10T06:52:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:10 smithi174 ceph-mon[39523]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-10T06:52:13.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:12 smithi184 ceph-mon[33513]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-10T06:52:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:12 smithi174 ceph-mon[29878]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-10T06:52:13.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:12 smithi174 ceph-mon[39523]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-02-10T06:52:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:14 smithi174 ceph-mon[29878]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-10T06:52:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:14 smithi174 ceph-mon[29878]: osdmap e396: 8 total, 8 up, 8 in 2024-02-10T06:52:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:14 smithi174 ceph-mon[39523]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-10T06:52:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:14 smithi174 ceph-mon[39523]: osdmap e396: 8 total, 8 up, 8 in 2024-02-10T06:52:15.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:14 smithi184 ceph-mon[33513]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-10T06:52:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:14 smithi184 ceph-mon[33513]: osdmap e396: 8 total, 8 up, 8 in 2024-02-10T06:52:16.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:52:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:52:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:16 smithi174 ceph-mon[29878]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:52:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:16 smithi174 ceph-mon[39523]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:52:17.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:16 smithi184 ceph-mon[33513]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:52:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:52:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:52:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:52:18.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:18 smithi174 ceph-mon[29878]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 205 B/s wr, 1 op/s 2024-02-10T06:52:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:18 smithi174 ceph-mon[29878]: osdmap e397: 8 total, 8 up, 8 in 2024-02-10T06:52:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:18 smithi174 ceph-mon[39523]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 205 B/s wr, 1 op/s 2024-02-10T06:52:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:18 smithi174 ceph-mon[39523]: osdmap e397: 8 total, 8 up, 8 in 2024-02-10T06:52:19.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:18 smithi184 ceph-mon[33513]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 205 B/s wr, 1 op/s 2024-02-10T06:52:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:18 smithi184 ceph-mon[33513]: osdmap e397: 8 total, 8 up, 8 in 2024-02-10T06:52:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:20 smithi174 ceph-mon[29878]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:52:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:20 smithi174 ceph-mon[39523]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:52:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:20 smithi184 ceph-mon[33513]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:52:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:21 smithi174 ceph-mon[29878]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:21 smithi174 ceph-mon[39523]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:22.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:21 smithi184 ceph-mon[33513]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:24.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:24 smithi184 ceph-mon[33513]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:24.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:24 smithi174 ceph-mon[29878]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:24 smithi174 ceph-mon[39523]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:26.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:25 smithi184 ceph-mon[33513]: osdmap e398: 8 total, 8 up, 8 in 2024-02-10T06:52:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:25 smithi174 ceph-mon[29878]: osdmap e398: 8 total, 8 up, 8 in 2024-02-10T06:52:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:25 smithi174 ceph-mon[39523]: osdmap e398: 8 total, 8 up, 8 in 2024-02-10T06:52:26.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:52:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:52:27.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:26 smithi174 ceph-mon[29878]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:27.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:26 smithi174 ceph-mon[39523]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:26 smithi184 ceph-mon[33513]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:52:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:52:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:52:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:28 smithi174 ceph-mon[29878]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:28 smithi174 ceph-mon[39523]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:29.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:28 smithi184 ceph-mon[33513]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:52:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:29 smithi174 ceph-mon[29878]: osdmap e399: 8 total, 8 up, 8 in 2024-02-10T06:52:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:29 smithi174 ceph-mon[39523]: osdmap e399: 8 total, 8 up, 8 in 2024-02-10T06:52:30.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:29 smithi184 ceph-mon[33513]: osdmap e399: 8 total, 8 up, 8 in 2024-02-10T06:52:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:30 smithi174 ceph-mon[29878]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:52:31.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:30 smithi174 ceph-mon[39523]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:52:31.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:30 smithi184 ceph-mon[33513]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:52:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:31 smithi174 ceph-mon[29878]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T06:52:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:31 smithi174 ceph-mon[39523]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T06:52:32.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:31 smithi184 ceph-mon[33513]: pgmap v564: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T06:52:34.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:34 smithi184 ceph-mon[33513]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:52:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:52:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:34 smithi174 ceph-mon[29878]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:52:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:52:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:34 smithi174 ceph-mon[39523]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:52:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:52:35.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:52:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:52:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:52:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:35 smithi184 ceph-mon[33513]: osdmap e400: 8 total, 8 up, 8 in 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[29878]: osdmap e400: 8 total, 8 up, 8 in 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:52:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:52:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:35 smithi174 ceph-mon[39523]: osdmap e400: 8 total, 8 up, 8 in 2024-02-10T06:52:36.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:52:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:52:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:36 smithi174 ceph-mon[29878]: pgmap v567: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:52:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:36 smithi174 ceph-mon[39523]: pgmap v567: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:52:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:36 smithi184 ceph-mon[33513]: pgmap v567: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:52:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:38.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:52:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:52:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:52:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:38 smithi174 ceph-mon[29878]: pgmap v568: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T06:52:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:38 smithi174 ceph-mon[39523]: pgmap v568: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T06:52:39.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:38 smithi184 ceph-mon[33513]: pgmap v568: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T06:52:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-63"}]: dispatch 2024-02-10T06:52:40.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-63"}]: dispatch 2024-02-10T06:52:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-63"}]: dispatch 2024-02-10T06:52:40.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-63"}]: dispatch 2024-02-10T06:52:40.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-63"}]: dispatch 2024-02-10T06:52:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-63"}]: dispatch 2024-02-10T06:52:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:40 smithi174 ceph-mon[29878]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T06:52:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:40 smithi174 ceph-mon[29878]: osdmap e401: 8 total, 8 up, 8 in 2024-02-10T06:52:41.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:40 smithi174 ceph-mon[39523]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T06:52:41.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:40 smithi174 ceph-mon[39523]: osdmap e401: 8 total, 8 up, 8 in 2024-02-10T06:52:41.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:40 smithi184 ceph-mon[33513]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T06:52:41.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:40 smithi184 ceph-mon[33513]: osdmap e401: 8 total, 8 up, 8 in 2024-02-10T06:52:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[29878]: osdmap e402: 8 total, 8 up, 8 in 2024-02-10T06:52:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[29878]: pgmap v572: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:52:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[29878]: osdmap e403: 8 total, 8 up, 8 in 2024-02-10T06:52:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[39523]: osdmap e402: 8 total, 8 up, 8 in 2024-02-10T06:52:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[39523]: pgmap v572: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:52:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:41 smithi174 ceph-mon[39523]: osdmap e403: 8 total, 8 up, 8 in 2024-02-10T06:52:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:41 smithi184 ceph-mon[33513]: osdmap e402: 8 total, 8 up, 8 in 2024-02-10T06:52:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:41 smithi184 ceph-mon[33513]: pgmap v572: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:41 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:52:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:41 smithi184 ceph-mon[33513]: osdmap e403: 8 total, 8 up, 8 in 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[29878]: osdmap e404: 8 total, 8 up, 8 in 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:52:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:42 smithi174 ceph-mon[39523]: osdmap e404: 8 total, 8 up, 8 in 2024-02-10T06:52:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:52:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:42 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:52:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:52:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:42 smithi184 ceph-mon[33513]: osdmap e404: 8 total, 8 up, 8 in 2024-02-10T06:52:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:43 smithi174 ceph-mon[29878]: pgmap v575: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:43 smithi174 ceph-mon[39523]: pgmap v575: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:44.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:43 smithi184 ceph-mon[33513]: pgmap v575: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:45.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:44 smithi174 ceph-mon[29878]: osdmap e405: 8 total, 8 up, 8 in 2024-02-10T06:52:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:44 smithi174 ceph-mon[39523]: osdmap e405: 8 total, 8 up, 8 in 2024-02-10T06:52:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:44 smithi184 ceph-mon[33513]: osdmap e405: 8 total, 8 up, 8 in 2024-02-10T06:52:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:45 smithi174 ceph-mon[29878]: osdmap e406: 8 total, 8 up, 8 in 2024-02-10T06:52:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:45 smithi174 ceph-mon[29878]: pgmap v578: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:45 smithi174 ceph-mon[39523]: osdmap e406: 8 total, 8 up, 8 in 2024-02-10T06:52:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:45 smithi174 ceph-mon[39523]: pgmap v578: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:46.310 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:45 smithi184 ceph-mon[33513]: osdmap e406: 8 total, 8 up, 8 in 2024-02-10T06:52:46.310 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:45 smithi184 ceph-mon[33513]: pgmap v578: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:46.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:52:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:52:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:46 smithi174 ceph-mon[29878]: osdmap e407: 8 total, 8 up, 8 in 2024-02-10T06:52:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:46 smithi174 ceph-mon[39523]: osdmap e407: 8 total, 8 up, 8 in 2024-02-10T06:52:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:46 smithi184 ceph-mon[33513]: osdmap e407: 8 total, 8 up, 8 in 2024-02-10T06:52:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:47 smithi174 ceph-mon[29878]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-02-10T06:52:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:47 smithi174 ceph-mon[39523]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-02-10T06:52:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:52:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:52:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:52:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:47 smithi184 ceph-mon[33513]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-02-10T06:52:50.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:50 smithi184 ceph-mon[33513]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-02-10T06:52:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:50 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:50 smithi174 ceph-mon[29878]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-02-10T06:52:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:50 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:50 smithi174 ceph-mon[39523]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-02-10T06:52:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:50 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:52:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:52 smithi184 ceph-mon[33513]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-10T06:52:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:52 smithi174 ceph-mon[29878]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-10T06:52:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:52 smithi174 ceph-mon[39523]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-10T06:52:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:52:54.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:54 smithi184 ceph-mon[33513]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T06:52:54.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:54 smithi174 ceph-mon[29878]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T06:52:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:54 smithi174 ceph-mon[39523]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T06:52:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:55 smithi174 ceph-mon[29878]: osdmap e408: 8 total, 8 up, 8 in 2024-02-10T06:52:56.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:55 smithi174 ceph-mon[39523]: osdmap e408: 8 total, 8 up, 8 in 2024-02-10T06:52:56.321 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:55 smithi184 ceph-mon[33513]: osdmap e408: 8 total, 8 up, 8 in 2024-02-10T06:52:56.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:52:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:52:57.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[29878]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-10T06:52:57.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:57.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:57.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-65"}]: dispatch 2024-02-10T06:52:57.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-65"}]: dispatch 2024-02-10T06:52:57.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[39523]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-10T06:52:57.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:57.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:57.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-65"}]: dispatch 2024-02-10T06:52:57.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-65"}]: dispatch 2024-02-10T06:52:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:56 smithi184 ceph-mon[33513]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-10T06:52:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:52:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-65"}]: dispatch 2024-02-10T06:52:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-65"}]: dispatch 2024-02-10T06:52:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:57 smithi174 ceph-mon[29878]: osdmap e409: 8 total, 8 up, 8 in 2024-02-10T06:52:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:52:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:52:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:52:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:57 smithi174 ceph-mon[39523]: osdmap e409: 8 total, 8 up, 8 in 2024-02-10T06:52:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:52:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:57 smithi184 ceph-mon[33513]: osdmap e409: 8 total, 8 up, 8 in 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[29878]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[29878]: osdmap e410: 8 total, 8 up, 8 in 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[39523]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[39523]: osdmap e410: 8 total, 8 up, 8 in 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:59.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:58 smithi184 ceph-mon[33513]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:52:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:58 smithi184 ceph-mon[33513]: osdmap e410: 8 total, 8 up, 8 in 2024-02-10T06:52:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:52:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[29878]: osdmap e411: 8 total, 8 up, 8 in 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[29878]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[39523]: osdmap e411: 8 total, 8 up, 8 in 2024-02-10T06:53:00.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:00.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:52:59 smithi174 ceph-mon[39523]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:00.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:59 smithi184 ceph-mon[33513]: osdmap e411: 8 total, 8 up, 8 in 2024-02-10T06:53:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:52:59 smithi184 ceph-mon[33513]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[29878]: osdmap e412: 8 total, 8 up, 8 in 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-67"}]: dispatch 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-67"}]: dispatch 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[39523]: osdmap e412: 8 total, 8 up, 8 in 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-67"}]: dispatch 2024-02-10T06:53:01.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-67"}]: dispatch 2024-02-10T06:53:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:00 smithi184 ceph-mon[33513]: osdmap e412: 8 total, 8 up, 8 in 2024-02-10T06:53:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-67"}]: dispatch 2024-02-10T06:53:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-67"}]: dispatch 2024-02-10T06:53:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:01 smithi174 ceph-mon[29878]: osdmap e413: 8 total, 8 up, 8 in 2024-02-10T06:53:02.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:01 smithi174 ceph-mon[29878]: pgmap v593: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:53:02.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:01 smithi174 ceph-mon[39523]: osdmap e413: 8 total, 8 up, 8 in 2024-02-10T06:53:02.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:01 smithi174 ceph-mon[39523]: pgmap v593: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:53:02.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:01 smithi184 ceph-mon[33513]: osdmap e413: 8 total, 8 up, 8 in 2024-02-10T06:53:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:01 smithi184 ceph-mon[33513]: pgmap v593: 196 pgs: 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:53:03.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:02 smithi174 ceph-mon[29878]: osdmap e414: 8 total, 8 up, 8 in 2024-02-10T06:53:03.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:03.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:03.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:02 smithi174 ceph-mon[39523]: osdmap e414: 8 total, 8 up, 8 in 2024-02-10T06:53:03.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:03.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:03.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:02 smithi184 ceph-mon[33513]: osdmap e414: 8 total, 8 up, 8 in 2024-02-10T06:53:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: osdmap e415: 8 total, 8 up, 8 in 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-69"}]: dispatch 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-69"}]: dispatch 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[29878]: pgmap v596: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: osdmap e415: 8 total, 8 up, 8 in 2024-02-10T06:53:04.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:04.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:04.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:04.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-69"}]: dispatch 2024-02-10T06:53:04.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-69"}]: dispatch 2024-02-10T06:53:04.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:03 smithi174 ceph-mon[39523]: pgmap v596: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: osdmap e415: 8 total, 8 up, 8 in 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-69"}]: dispatch 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-69"}]: dispatch 2024-02-10T06:53:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:03 smithi184 ceph-mon[33513]: pgmap v596: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T06:53:05.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:04 smithi174 ceph-mon[29878]: osdmap e416: 8 total, 8 up, 8 in 2024-02-10T06:53:05.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:04 smithi174 ceph-mon[39523]: osdmap e416: 8 total, 8 up, 8 in 2024-02-10T06:53:05.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:04 smithi184 ceph-mon[33513]: osdmap e416: 8 total, 8 up, 8 in 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[29878]: osdmap e417: 8 total, 8 up, 8 in 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[29878]: pgmap v599: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[39523]: osdmap e417: 8 total, 8 up, 8 in 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:06.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:06.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:05 smithi174 ceph-mon[39523]: pgmap v599: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:05 smithi184 ceph-mon[33513]: osdmap e417: 8 total, 8 up, 8 in 2024-02-10T06:53:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:05 smithi184 ceph-mon[33513]: pgmap v599: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:06.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: osdmap e418: 8 total, 8 up, 8 in 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:07.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: osdmap e418: 8 total, 8 up, 8 in 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:07.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:06 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: osdmap e418: 8 total, 8 up, 8 in 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:06 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:53:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-mon[29878]: osdmap e419: 8 total, 8 up, 8 in 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-mon[29878]: pgmap v602: 228 pgs: 3 creating+activating, 25 creating+peering, 200 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-mon[39523]: osdmap e419: 8 total, 8 up, 8 in 2024-02-10T06:53:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:07 smithi174 ceph-mon[39523]: pgmap v602: 228 pgs: 3 creating+activating, 25 creating+peering, 200 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:53:08.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:53:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:07 smithi184 ceph-mon[33513]: osdmap e419: 8 total, 8 up, 8 in 2024-02-10T06:53:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:07 smithi184 ceph-mon[33513]: pgmap v602: 228 pgs: 3 creating+activating, 25 creating+peering, 200 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:53:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:08 smithi174 ceph-mon[29878]: osdmap e420: 8 total, 8 up, 8 in 2024-02-10T06:53:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:08 smithi174 ceph-mon[39523]: osdmap e420: 8 total, 8 up, 8 in 2024-02-10T06:53:09.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:08 smithi184 ceph-mon[33513]: osdmap e420: 8 total, 8 up, 8 in 2024-02-10T06:53:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:10 smithi174 ceph-mon[29878]: pgmap v604: 228 pgs: 3 creating+activating, 25 creating+peering, 200 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 754 B/s wr, 2 op/s 2024-02-10T06:53:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:10 smithi174 ceph-mon[39523]: pgmap v604: 228 pgs: 3 creating+activating, 25 creating+peering, 200 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 754 B/s wr, 2 op/s 2024-02-10T06:53:11.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:10 smithi184 ceph-mon[33513]: pgmap v604: 228 pgs: 3 creating+activating, 25 creating+peering, 200 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 754 B/s wr, 2 op/s 2024-02-10T06:53:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:12 smithi174 ceph-mon[29878]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-10T06:53:13.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:12 smithi174 ceph-mon[39523]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-10T06:53:13.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:12 smithi184 ceph-mon[33513]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-10T06:53:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:14 smithi174 ceph-mon[29878]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-10T06:53:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:14 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:14 smithi174 ceph-mon[39523]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-10T06:53:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:14 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:15.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:14 smithi184 ceph-mon[33513]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-10T06:53:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:14 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:16.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:53:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:53:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:16 smithi174 ceph-mon[29878]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 762 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-10T06:53:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:16 smithi174 ceph-mon[39523]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 762 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-10T06:53:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:16 smithi184 ceph-mon[33513]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 762 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-10T06:53:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:53:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:53:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:53:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:18.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:18 smithi174 ceph-mon[29878]: pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-10T06:53:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:18 smithi174 ceph-mon[39523]: pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-10T06:53:19.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:18 smithi184 ceph-mon[33513]: pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-10T06:53:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:20 smithi174 ceph-mon[29878]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1019 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:53:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:20 smithi174 ceph-mon[39523]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1019 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:53:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:20 smithi184 ceph-mon[33513]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1019 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:53:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:22.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:22 smithi174 ceph-mon[29878]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T06:53:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:22 smithi174 ceph-mon[39523]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T06:53:23.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:22 smithi184 ceph-mon[33513]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T06:53:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:23 smithi174 ceph-mon[29878]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:23 smithi174 ceph-mon[39523]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:24.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:23 smithi184 ceph-mon[33513]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:26.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:53:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:53:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:26 smithi174 ceph-mon[29878]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:26 smithi174 ceph-mon[39523]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:26 smithi184 ceph-mon[33513]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:28.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:28.141 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:53:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:53:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:53:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[39523]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-71"}]: dispatch 2024-02-10T06:53:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-71"}]: dispatch 2024-02-10T06:53:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[29878]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:29.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:29.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:29.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-71"}]: dispatch 2024-02-10T06:53:29.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-71"}]: dispatch 2024-02-10T06:53:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:28 smithi184 ceph-mon[33513]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-71"}]: dispatch 2024-02-10T06:53:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-71"}]: dispatch 2024-02-10T06:53:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:29 smithi174 ceph-mon[29878]: osdmap e421: 8 total, 8 up, 8 in 2024-02-10T06:53:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:29 smithi174 ceph-mon[39523]: osdmap e421: 8 total, 8 up, 8 in 2024-02-10T06:53:30.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:29 smithi184 ceph-mon[33513]: osdmap e421: 8 total, 8 up, 8 in 2024-02-10T06:53:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[29878]: pgmap v615: 196 pgs: 196 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-10T06:53:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[29878]: osdmap e422: 8 total, 8 up, 8 in 2024-02-10T06:53:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:31.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[39523]: pgmap v615: 196 pgs: 196 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-10T06:53:31.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[39523]: osdmap e422: 8 total, 8 up, 8 in 2024-02-10T06:53:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:30 smithi184 ceph-mon[33513]: pgmap v615: 196 pgs: 196 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-10T06:53:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:30 smithi184 ceph-mon[33513]: osdmap e422: 8 total, 8 up, 8 in 2024-02-10T06:53:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[29878]: osdmap e423: 8 total, 8 up, 8 in 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[29878]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[39523]: osdmap e423: 8 total, 8 up, 8 in 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:31 smithi174 ceph-mon[39523]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:32.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:31 smithi184 ceph-mon[33513]: osdmap e423: 8 total, 8 up, 8 in 2024-02-10T06:53:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:31 smithi184 ceph-mon[33513]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:32 smithi174 ceph-mon[29878]: osdmap e424: 8 total, 8 up, 8 in 2024-02-10T06:53:33.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:32 smithi174 ceph-mon[39523]: osdmap e424: 8 total, 8 up, 8 in 2024-02-10T06:53:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:32 smithi184 ceph-mon[33513]: osdmap e424: 8 total, 8 up, 8 in 2024-02-10T06:53:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:33 smithi174 ceph-mon[29878]: osdmap e425: 8 total, 8 up, 8 in 2024-02-10T06:53:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:33 smithi174 ceph-mon[29878]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:53:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:33 smithi174 ceph-mon[39523]: osdmap e425: 8 total, 8 up, 8 in 2024-02-10T06:53:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:33 smithi174 ceph-mon[39523]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:53:34.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:33 smithi184 ceph-mon[33513]: osdmap e425: 8 total, 8 up, 8 in 2024-02-10T06:53:34.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:33 smithi184 ceph-mon[33513]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:53:35.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:34 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:34 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:35.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:34 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[29878]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 996 B/s rd, 996 B/s wr, 1 op/s 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:53:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:35 smithi174 ceph-mon[39523]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 996 B/s rd, 996 B/s wr, 1 op/s 2024-02-10T06:53:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:53:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:53:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:53:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:35 smithi184 ceph-mon[33513]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 996 B/s rd, 996 B/s wr, 1 op/s 2024-02-10T06:53:36.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:53:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:53:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:36 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:36 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:37.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:36 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:37 smithi174 ceph-mon[29878]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-02-10T06:53:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:53:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:53:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:53:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:37 smithi174 ceph-mon[39523]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-02-10T06:53:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:37 smithi184 ceph-mon[33513]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-02-10T06:53:41.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:40 smithi174 ceph-mon[29878]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:41.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:40 smithi174 ceph-mon[39523]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:41.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:40 smithi184 ceph-mon[33513]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:53:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:42 smithi174 ceph-mon[29878]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 224 B/s wr, 2 op/s 2024-02-10T06:53:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:42 smithi174 ceph-mon[39523]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 224 B/s wr, 2 op/s 2024-02-10T06:53:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:42 smithi184 ceph-mon[33513]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 224 B/s wr, 2 op/s 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-73"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-73"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-73"}]: dispatch 2024-02-10T06:53:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-73"}]: dispatch 2024-02-10T06:53:44.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-73"}]: dispatch 2024-02-10T06:53:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-73"}]: dispatch 2024-02-10T06:53:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:44 smithi174 ceph-mon[29878]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 202 B/s wr, 2 op/s 2024-02-10T06:53:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:44 smithi174 ceph-mon[29878]: osdmap e426: 8 total, 8 up, 8 in 2024-02-10T06:53:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:44 smithi174 ceph-mon[39523]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 202 B/s wr, 2 op/s 2024-02-10T06:53:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:44 smithi174 ceph-mon[39523]: osdmap e426: 8 total, 8 up, 8 in 2024-02-10T06:53:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:44 smithi184 ceph-mon[33513]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 202 B/s wr, 2 op/s 2024-02-10T06:53:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:44 smithi184 ceph-mon[33513]: osdmap e426: 8 total, 8 up, 8 in 2024-02-10T06:53:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[29878]: osdmap e427: 8 total, 8 up, 8 in 2024-02-10T06:53:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[29878]: pgmap v629: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[29878]: osdmap e428: 8 total, 8 up, 8 in 2024-02-10T06:53:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[39523]: osdmap e427: 8 total, 8 up, 8 in 2024-02-10T06:53:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[39523]: pgmap v629: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:46.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:45 smithi174 ceph-mon[39523]: osdmap e428: 8 total, 8 up, 8 in 2024-02-10T06:53:46.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:45 smithi184 ceph-mon[33513]: osdmap e427: 8 total, 8 up, 8 in 2024-02-10T06:53:46.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:46.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:46.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:45 smithi184 ceph-mon[33513]: pgmap v629: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:46.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:46.359 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:45 smithi184 ceph-mon[33513]: osdmap e428: 8 total, 8 up, 8 in 2024-02-10T06:53:46.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:53:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[29878]: osdmap e429: 8 total, 8 up, 8 in 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:46 smithi174 ceph-mon[39523]: osdmap e429: 8 total, 8 up, 8 in 2024-02-10T06:53:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:46 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:46 smithi184 ceph-mon[33513]: osdmap e429: 8 total, 8 up, 8 in 2024-02-10T06:53:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:47 smithi174 ceph-mon[29878]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:53:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:53:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:53:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:47 smithi174 ceph-mon[39523]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:47 smithi184 ceph-mon[33513]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[39523]: osdmap e430: 8 total, 8 up, 8 in 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-75"}]: dispatch 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-75"}]: dispatch 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[29878]: osdmap e430: 8 total, 8 up, 8 in 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:49.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-75"}]: dispatch 2024-02-10T06:53:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-75"}]: dispatch 2024-02-10T06:53:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:48 smithi184 ceph-mon[33513]: osdmap e430: 8 total, 8 up, 8 in 2024-02-10T06:53:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-75"}]: dispatch 2024-02-10T06:53:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-75"}]: dispatch 2024-02-10T06:53:50.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:49 smithi174 ceph-mon[39523]: osdmap e431: 8 total, 8 up, 8 in 2024-02-10T06:53:50.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:49 smithi174 ceph-mon[39523]: pgmap v635: 196 pgs: 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:53:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:49 smithi174 ceph-mon[29878]: osdmap e431: 8 total, 8 up, 8 in 2024-02-10T06:53:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:49 smithi174 ceph-mon[29878]: pgmap v635: 196 pgs: 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:53:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:49 smithi184 ceph-mon[33513]: osdmap e431: 8 total, 8 up, 8 in 2024-02-10T06:53:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:49 smithi184 ceph-mon[33513]: pgmap v635: 196 pgs: 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:53:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:50 smithi174 ceph-mon[39523]: osdmap e432: 8 total, 8 up, 8 in 2024-02-10T06:53:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:50 smithi174 ceph-mon[29878]: osdmap e432: 8 total, 8 up, 8 in 2024-02-10T06:53:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:51.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:50 smithi184 ceph-mon[33513]: osdmap e432: 8 total, 8 up, 8 in 2024-02-10T06:53:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[39523]: osdmap e433: 8 total, 8 up, 8 in 2024-02-10T06:53:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[39523]: pgmap v638: 228 pgs: 6 creating+activating, 222 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:53:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:52.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:52.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[29878]: osdmap e433: 8 total, 8 up, 8 in 2024-02-10T06:53:52.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:52.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[29878]: pgmap v638: 228 pgs: 6 creating+activating, 222 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:53:52.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:52.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:51 smithi184 ceph-mon[33513]: osdmap e433: 8 total, 8 up, 8 in 2024-02-10T06:53:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:51 smithi184 ceph-mon[33513]: pgmap v638: 228 pgs: 6 creating+activating, 222 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:53:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:53:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[29878]: osdmap e434: 8 total, 8 up, 8 in 2024-02-10T06:53:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-77"}]: dispatch 2024-02-10T06:53:53.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-77"}]: dispatch 2024-02-10T06:53:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[39523]: osdmap e434: 8 total, 8 up, 8 in 2024-02-10T06:53:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-77"}]: dispatch 2024-02-10T06:53:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-77"}]: dispatch 2024-02-10T06:53:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:52 smithi184 ceph-mon[33513]: osdmap e434: 8 total, 8 up, 8 in 2024-02-10T06:53:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:53:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-77"}]: dispatch 2024-02-10T06:53:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-77"}]: dispatch 2024-02-10T06:53:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:53 smithi174 ceph-mon[29878]: osdmap e435: 8 total, 8 up, 8 in 2024-02-10T06:53:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:53 smithi174 ceph-mon[29878]: pgmap v641: 196 pgs: 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:53:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:53 smithi174 ceph-mon[39523]: osdmap e435: 8 total, 8 up, 8 in 2024-02-10T06:53:54.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:53 smithi174 ceph-mon[39523]: pgmap v641: 196 pgs: 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:53:54.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:53 smithi184 ceph-mon[33513]: osdmap e435: 8 total, 8 up, 8 in 2024-02-10T06:53:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:53 smithi184 ceph-mon[33513]: pgmap v641: 196 pgs: 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:53:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[29878]: osdmap e436: 8 total, 8 up, 8 in 2024-02-10T06:53:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[39523]: osdmap e436: 8 total, 8 up, 8 in 2024-02-10T06:53:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:55.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:54 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:54 smithi184 ceph-mon[33513]: osdmap e436: 8 total, 8 up, 8 in 2024-02-10T06:53:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:53:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:54 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:53:56.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:56.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:56 smithi184 ceph-mon[33513]: osdmap e437: 8 total, 8 up, 8 in 2024-02-10T06:53:56.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:56.370 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:56 smithi184 ceph-mon[33513]: pgmap v644: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[39523]: osdmap e437: 8 total, 8 up, 8 in 2024-02-10T06:53:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[39523]: pgmap v644: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:56.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:53:56.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[29878]: osdmap e437: 8 total, 8 up, 8 in 2024-02-10T06:53:56.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:53:56.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:56 smithi174 ceph-mon[29878]: pgmap v644: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:53:56.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:53:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:53:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:57 smithi184 ceph-mon[33513]: osdmap e438: 8 total, 8 up, 8 in 2024-02-10T06:53:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:57 smithi174 ceph-mon[29878]: osdmap e438: 8 total, 8 up, 8 in 2024-02-10T06:53:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:57 smithi174 ceph-mon[39523]: osdmap e438: 8 total, 8 up, 8 in 2024-02-10T06:53:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:53:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:53:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[29878]: osdmap e439: 8 total, 8 up, 8 in 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.f"}]: dispatch 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[29878]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.f"}]: dispatch 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[29878]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[39523]: osdmap e439: 8 total, 8 up, 8 in 2024-02-10T06:53:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.f"}]: dispatch 2024-02-10T06:53:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[39523]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.f"}]: dispatch 2024-02-10T06:53:58.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:58 smithi174 ceph-mon[39523]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:53:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:58 smithi184 ceph-mon[33513]: osdmap e439: 8 total, 8 up, 8 in 2024-02-10T06:53:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.f"}]: dispatch 2024-02-10T06:53:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:58 smithi184 ceph-mon[33513]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.f"}]: dispatch 2024-02-10T06:53:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:58 smithi184 ceph-mon[33513]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:53:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:59 smithi184 ceph-mon[33513]: 305.f deep-scrub starts 2024-02-10T06:53:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:53:59 smithi184 ceph-mon[33513]: 305.f deep-scrub ok 2024-02-10T06:53:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:59 smithi174 ceph-mon[29878]: 305.f deep-scrub starts 2024-02-10T06:53:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:53:59 smithi174 ceph-mon[29878]: 305.f deep-scrub ok 2024-02-10T06:53:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:59 smithi174 ceph-mon[39523]: 305.f deep-scrub starts 2024-02-10T06:53:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:53:59 smithi174 ceph-mon[39523]: 305.f deep-scrub ok 2024-02-10T06:54:01.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:00 smithi174 ceph-mon[29878]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1010 B/s wr, 2 op/s 2024-02-10T06:54:01.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:00 smithi174 ceph-mon[39523]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1010 B/s wr, 2 op/s 2024-02-10T06:54:01.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:00 smithi184 ceph-mon[33513]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1010 B/s wr, 2 op/s 2024-02-10T06:54:03.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:02 smithi174 ceph-mon[29878]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T06:54:03.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:02 smithi174 ceph-mon[39523]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T06:54:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:02 smithi184 ceph-mon[33513]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T06:54:05.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:04 smithi174 ceph-mon[29878]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-10T06:54:05.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:04 smithi174 ceph-mon[39523]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-10T06:54:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:04 smithi184 ceph-mon[33513]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-10T06:54:06.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:54:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:54:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:06 smithi174 ceph-mon[29878]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 226 B/s wr, 1 op/s 2024-02-10T06:54:07.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:07.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:06 smithi174 ceph-mon[39523]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 226 B/s wr, 1 op/s 2024-02-10T06:54:07.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:07.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:06 smithi184 ceph-mon[33513]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 226 B/s wr, 1 op/s 2024-02-10T06:54:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:54:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:54:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:54:08.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:09.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:08 smithi174 ceph-mon[29878]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 203 B/s wr, 1 op/s 2024-02-10T06:54:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:08 smithi174 ceph-mon[39523]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 203 B/s wr, 1 op/s 2024-02-10T06:54:09.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:08 smithi184 ceph-mon[33513]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1018 B/s rd, 203 B/s wr, 1 op/s 2024-02-10T06:54:11.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:10 smithi174 ceph-mon[29878]: pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-02-10T06:54:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:10 smithi174 ceph-mon[39523]: pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-02-10T06:54:11.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:10 smithi184 ceph-mon[33513]: pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-02-10T06:54:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:11 smithi174 ceph-mon[29878]: pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 170 B/s wr, 61 op/s 2024-02-10T06:54:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:11 smithi174 ceph-mon[39523]: pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 170 B/s wr, 61 op/s 2024-02-10T06:54:12.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:11 smithi184 ceph-mon[33513]: pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 170 B/s wr, 61 op/s 2024-02-10T06:54:14.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:14 smithi184 ceph-mon[33513]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:14.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:14 smithi174 ceph-mon[29878]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:14 smithi174 ceph-mon[39523]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:16.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:54:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:54:17.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:16 smithi174 ceph-mon[29878]: pgmap v656: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:16 smithi174 ceph-mon[39523]: pgmap v656: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:17.239 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2[56431]: 2024-02-10T06:54:17.056+0000 7f8198cc4700 -1 snap_mapper.add_oid found existing snaps mapped on 305:f68ff0ca:test-rados-api-smithi174-79953-80::foo:2, removing 2024-02-10T06:54:17.240 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T06:54:17.056+0000 7fac5e9c2700 -1 snap_mapper.add_oid found existing snaps mapped on 305:f68ff0ca:test-rados-api-smithi174-79953-80::foo:2, removing 2024-02-10T06:54:17.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:16 smithi184 ceph-mon[33513]: pgmap v656: 228 pgs: 228 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:17.373 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 06:54:17 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T06:54:17.055+0000 7f378eb78700 -1 snap_mapper.add_oid found existing snaps mapped on 305:f68ff0ca:test-rados-api-smithi174-79953-80::foo:2, removing 2024-02-10T06:54:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:54:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:54:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-79"}]: dispatch 2024-02-10T06:54:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-79"}]: dispatch 2024-02-10T06:54:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-79"}]: dispatch 2024-02-10T06:54:18.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:17 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-79"}]: dispatch 2024-02-10T06:54:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-79"}]: dispatch 2024-02-10T06:54:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:17 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-79"}]: dispatch 2024-02-10T06:54:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:18 smithi174 ceph-mon[29878]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:18 smithi174 ceph-mon[29878]: osdmap e440: 8 total, 8 up, 8 in 2024-02-10T06:54:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:18 smithi174 ceph-mon[39523]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:18 smithi174 ceph-mon[39523]: osdmap e440: 8 total, 8 up, 8 in 2024-02-10T06:54:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:18 smithi184 ceph-mon[33513]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T06:54:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:18 smithi184 ceph-mon[33513]: osdmap e440: 8 total, 8 up, 8 in 2024-02-10T06:54:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:19 smithi174 ceph-mon[29878]: osdmap e441: 8 total, 8 up, 8 in 2024-02-10T06:54:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:19 smithi174 ceph-mon[39523]: osdmap e441: 8 total, 8 up, 8 in 2024-02-10T06:54:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:20.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:19 smithi184 ceph-mon[33513]: osdmap e441: 8 total, 8 up, 8 in 2024-02-10T06:54:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: pgmap v660: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: osdmap e442: 8 total, 8 up, 8 in 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: pgmap v660: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: osdmap e442: 8 total, 8 up, 8 in 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:20 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: pgmap v660: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: osdmap e442: 8 total, 8 up, 8 in 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:20 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[29878]: osdmap e443: 8 total, 8 up, 8 in 2024-02-10T06:54:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[29878]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-10T06:54:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[39523]: osdmap e443: 8 total, 8 up, 8 in 2024-02-10T06:54:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[39523]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-10T06:54:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:21 smithi184 ceph-mon[33513]: osdmap e443: 8 total, 8 up, 8 in 2024-02-10T06:54:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:22.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:21 smithi184 ceph-mon[33513]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-10T06:54:22.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: osdmap e444: 8 total, 8 up, 8 in 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: osdmap e445: 8 total, 8 up, 8 in 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: osdmap e444: 8 total, 8 up, 8 in 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: osdmap e445: 8 total, 8 up, 8 in 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:23.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:22 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: osdmap e444: 8 total, 8 up, 8 in 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: osdmap e445: 8 total, 8 up, 8 in 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:22 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:23 smithi174 ceph-mon[29878]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:54:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:24.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:23 smithi174 ceph-mon[29878]: osdmap e446: 8 total, 8 up, 8 in 2024-02-10T06:54:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:23 smithi174 ceph-mon[39523]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:54:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:23 smithi174 ceph-mon[39523]: osdmap e446: 8 total, 8 up, 8 in 2024-02-10T06:54:24.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:23 smithi184 ceph-mon[33513]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:54:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:23 smithi184 ceph-mon[33513]: osdmap e446: 8 total, 8 up, 8 in 2024-02-10T06:54:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T06:54:25.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T06:54:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T06:54:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T06:54:25.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T06:54:25.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[29878]: osdmap e447: 8 total, 8 up, 8 in 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[29878]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[39523]: osdmap e447: 8 total, 8 up, 8 in 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T06:54:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T06:54:26.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:25 smithi174 ceph-mon[39523]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T06:54:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:25 smithi184 ceph-mon[33513]: osdmap e447: 8 total, 8 up, 8 in 2024-02-10T06:54:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T06:54:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T06:54:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:25 smithi184 ceph-mon[33513]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:26.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:54:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[29878]: osdmap e448: 8 total, 8 up, 8 in 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T06:54:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[39523]: osdmap e448: 8 total, 8 up, 8 in 2024-02-10T06:54:27.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:27.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:27.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:26 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:27.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T06:54:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:26 smithi184 ceph-mon[33513]: osdmap e448: 8 total, 8 up, 8 in 2024-02-10T06:54:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:26 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:28.140 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:54:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:54:28.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:28.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: osdmap e449: 8 total, 8 up, 8 in 2024-02-10T06:54:28.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:28.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-81"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-81"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[29878]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 6.5 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: osdmap e449: 8 total, 8 up, 8 in 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-81"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-81"}]: dispatch 2024-02-10T06:54:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:27 smithi174 ceph-mon[39523]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 6.5 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: osdmap e449: 8 total, 8 up, 8 in 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-81"}]: dispatch 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-81"}]: dispatch 2024-02-10T06:54:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:27 smithi184 ceph-mon[33513]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 6.5 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-02-10T06:54:29.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:28 smithi174 ceph-mon[29878]: osdmap e450: 8 total, 8 up, 8 in 2024-02-10T06:54:29.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:28 smithi174 ceph-mon[39523]: osdmap e450: 8 total, 8 up, 8 in 2024-02-10T06:54:29.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:28 smithi184 ceph-mon[33513]: osdmap e450: 8 total, 8 up, 8 in 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[29878]: osdmap e451: 8 total, 8 up, 8 in 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[29878]: pgmap v675: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 6.5 KiB/s rd, 9.0 KiB/s wr, 26 op/s 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[39523]: osdmap e451: 8 total, 8 up, 8 in 2024-02-10T06:54:30.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[39523]: pgmap v675: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 6.5 KiB/s rd, 9.0 KiB/s wr, 26 op/s 2024-02-10T06:54:30.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:29 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:30.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:29 smithi184 ceph-mon[33513]: osdmap e451: 8 total, 8 up, 8 in 2024-02-10T06:54:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:29 smithi184 ceph-mon[33513]: pgmap v675: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 6.5 KiB/s rd, 9.0 KiB/s wr, 26 op/s 2024-02-10T06:54:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:29 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:31.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:30 smithi184 ceph-mon[33513]: osdmap e452: 8 total, 8 up, 8 in 2024-02-10T06:54:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[29878]: osdmap e452: 8 total, 8 up, 8 in 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[39523]: osdmap e452: 8 total, 8 up, 8 in 2024-02-10T06:54:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:31.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: osdmap e453: 8 total, 8 up, 8 in 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1023 B/s wr, 13 op/s 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: osdmap e454: 8 total, 8 up, 8 in 2024-02-10T06:54:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: osdmap e453: 8 total, 8 up, 8 in 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1023 B/s wr, 13 op/s 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: osdmap e454: 8 total, 8 up, 8 in 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: osdmap e453: 8 total, 8 up, 8 in 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 1023 B/s wr, 13 op/s 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: osdmap e454: 8 total, 8 up, 8 in 2024-02-10T06:54:32.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:32 smithi184 ceph-mon[33513]: osdmap e455: 8 total, 8 up, 8 in 2024-02-10T06:54:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:32 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[29878]: osdmap e455: 8 total, 8 up, 8 in 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[39523]: osdmap e455: 8 total, 8 up, 8 in 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:32 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:34.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:34 smithi184 ceph-mon[33513]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-02-10T06:54:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:34 smithi184 ceph-mon[33513]: osdmap e456: 8 total, 8 up, 8 in 2024-02-10T06:54:34.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:34 smithi174 ceph-mon[29878]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-02-10T06:54:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:34 smithi174 ceph-mon[29878]: osdmap e456: 8 total, 8 up, 8 in 2024-02-10T06:54:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:34 smithi174 ceph-mon[39523]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail; 4.7 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-02-10T06:54:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:34 smithi174 ceph-mon[39523]: osdmap e456: 8 total, 8 up, 8 in 2024-02-10T06:54:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-mon[33513]: osdmap e457: 8 total, 8 up, 8 in 2024-02-10T06:54:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-mon[33513]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:36.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[29878]: osdmap e457: 8 total, 8 up, 8 in 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[29878]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[39523]: osdmap e457: 8 total, 8 up, 8 in 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[39523]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 671 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:54:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:54:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:54:36.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:54:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: osdmap e458: 8 total, 8 up, 8 in 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-83"}]: dispatch 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-83"}]: dispatch 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: osdmap e458: 8 total, 8 up, 8 in 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-83"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-83"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: osdmap e458: 8 total, 8 up, 8 in 2024-02-10T06:54:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-83"}]: dispatch 2024-02-10T06:54:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-83"}]: dispatch 2024-02-10T06:54:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:38 smithi174 ceph-mon[29878]: osdmap e459: 8 total, 8 up, 8 in 2024-02-10T06:54:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:38 smithi174 ceph-mon[29878]: pgmap v687: 196 pgs: 196 active+clean; 8.4 MiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:38.240 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:54:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:54:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:54:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:38 smithi174 ceph-mon[39523]: osdmap e459: 8 total, 8 up, 8 in 2024-02-10T06:54:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:38 smithi174 ceph-mon[39523]: pgmap v687: 196 pgs: 196 active+clean; 8.4 MiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:38 smithi184 ceph-mon[33513]: osdmap e459: 8 total, 8 up, 8 in 2024-02-10T06:54:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:38 smithi184 ceph-mon[33513]: pgmap v687: 196 pgs: 196 active+clean; 8.4 MiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:39 smithi184 ceph-mon[33513]: osdmap e460: 8 total, 8 up, 8 in 2024-02-10T06:54:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:39 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:39 smithi174 ceph-mon[29878]: osdmap e460: 8 total, 8 up, 8 in 2024-02-10T06:54:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:39 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:39 smithi174 ceph-mon[39523]: osdmap e460: 8 total, 8 up, 8 in 2024-02-10T06:54:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:39 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: osdmap e461: 8 total, 8 up, 8 in 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: osdmap e462: 8 total, 8 up, 8 in 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:40 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: osdmap e461: 8 total, 8 up, 8 in 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: osdmap e462: 8 total, 8 up, 8 in 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: osdmap e461: 8 total, 8 up, 8 in 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 675 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: osdmap e462: 8 total, 8 up, 8 in 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:40.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:40 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:42 smithi184 ceph-mon[33513]: osdmap e463: 8 total, 8 up, 8 in 2024-02-10T06:54:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:42 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:42 smithi184 ceph-mon[33513]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[29878]: osdmap e463: 8 total, 8 up, 8 in 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[29878]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_tier","val": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:54:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[39523]: osdmap e463: 8 total, 8 up, 8 in 2024-02-10T06:54:42.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:42.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:42.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:42 smithi174 ceph-mon[39523]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-10T06:54:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:43 smithi184 ceph-mon[33513]: osdmap e464: 8 total, 8 up, 8 in 2024-02-10T06:54:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:43 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[29878]: osdmap e464: 8 total, 8 up, 8 in 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[39523]: osdmap e464: 8 total, 8 up, 8 in 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:43 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:44 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:44 smithi184 ceph-mon[33513]: osdmap e465: 8 total, 8 up, 8 in 2024-02-10T06:54:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:44 smithi184 ceph-mon[33513]: pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-10T06:54:44.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:44 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:44 smithi174 ceph-mon[29878]: osdmap e465: 8 total, 8 up, 8 in 2024-02-10T06:54:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:44 smithi174 ceph-mon[29878]: pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-10T06:54:44.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:44 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:44.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:44 smithi174 ceph-mon[39523]: osdmap e465: 8 total, 8 up, 8 in 2024-02-10T06:54:44.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:44 smithi174 ceph-mon[39523]: pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-10T06:54:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:45 smithi184 ceph-mon[33513]: osdmap e466: 8 total, 8 up, 8 in 2024-02-10T06:54:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:45 smithi174 ceph-mon[29878]: osdmap e466: 8 total, 8 up, 8 in 2024-02-10T06:54:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:45 smithi174 ceph-mon[39523]: osdmap e466: 8 total, 8 up, 8 in 2024-02-10T06:54:46.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: osdmap e467: 8 total, 8 up, 8 in 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "308.5", "id": [5, 2]}]: dispatch 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-85"}]: dispatch 2024-02-10T06:54:46.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-85"}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: osdmap e467: 8 total, 8 up, 8 in 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "308.5", "id": [5, 2]}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-85"}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-85"}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: osdmap e467: 8 total, 8 up, 8 in 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "308.5", "id": [5, 2]}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:46.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-85"}]: dispatch 2024-02-10T06:54:46.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-85"}]: dispatch 2024-02-10T06:54:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:47 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "308.5", "id": [5, 2]}]': finished 2024-02-10T06:54:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:47 smithi184 ceph-mon[33513]: osdmap e468: 8 total, 8 up, 8 in 2024-02-10T06:54:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:47 smithi184 ceph-mon[33513]: osdmap e469: 8 total, 8 up, 8 in 2024-02-10T06:54:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "308.5", "id": [5, 2]}]': finished 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[29878]: osdmap e468: 8 total, 8 up, 8 in 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[29878]: osdmap e469: 8 total, 8 up, 8 in 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "308.5", "id": [5, 2]}]': finished 2024-02-10T06:54:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[39523]: osdmap e468: 8 total, 8 up, 8 in 2024-02-10T06:54:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[39523]: osdmap e469: 8 total, 8 up, 8 in 2024-02-10T06:54:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:47.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18073 ms) 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22696 ms) 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3015 ms) 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3034 ms) 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-10T06:54:48.083 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3026 ms) 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3018 ms) 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4041 ms) 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39871 ms) 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17016 ms) 2024-02-10T06:54:48.084 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4026 ms) 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3024 ms) 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24942 ms) 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14996 ms) 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5049 ms) 2024-02-10T06:54:48.085 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4031 ms) 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: waiting for scrubs... 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: done waiting 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24928 ms) 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10069 ms) 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9052 ms) 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:54:48.086 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9043 ms) 2024-02-10T06:54:48.087 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-10T06:54:48.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:54:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:54:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:54:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:48 smithi184 ceph-mon[33513]: pgmap v702: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T06:54:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:48 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:48 smithi184 ceph-mon[33513]: osdmap e470: 8 total, 8 up, 8 in 2024-02-10T06:54:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:48.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[29878]: pgmap v702: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[29878]: osdmap e470: 8 total, 8 up, 8 in 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[39523]: pgmap v702: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[39523]: osdmap e470: 8 total, 8 up, 8 in 2024-02-10T06:54:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:50.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:50 smithi184 ceph-mon[33513]: pgmap v704: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T06:54:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:50 smithi184 ceph-mon[33513]: osdmap e471: 8 total, 8 up, 8 in 2024-02-10T06:54:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[29878]: pgmap v704: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T06:54:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[29878]: osdmap e471: 8 total, 8 up, 8 in 2024-02-10T06:54:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[39523]: pgmap v704: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T06:54:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[39523]: osdmap e471: 8 total, 8 up, 8 in 2024-02-10T06:54:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:51 smithi184 ceph-mon[33513]: osdmap e472: 8 total, 8 up, 8 in 2024-02-10T06:54:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]: dispatch 2024-02-10T06:54:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]: dispatch 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[29878]: osdmap e472: 8 total, 8 up, 8 in 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]: dispatch 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]: dispatch 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[39523]: osdmap e472: 8 total, 8 up, 8 in 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]: dispatch 2024-02-10T06:54:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]: dispatch 2024-02-10T06:54:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:52 smithi184 ceph-mon[33513]: pgmap v707: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:54:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]': finished 2024-02-10T06:54:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:52 smithi184 ceph-mon[33513]: osdmap e473: 8 total, 8 up, 8 in 2024-02-10T06:54:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[29878]: pgmap v707: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]': finished 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[29878]: osdmap e473: 8 total, 8 up, 8 in 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[39523]: pgmap v707: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_tier","val": "test-rados-api-smithi174-79953-89-test-flush"}]': finished 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[39523]: osdmap e473: 8 total, 8 up, 8 in 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:54:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:54:53.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:53 smithi184 ceph-mon[33513]: osdmap e474: 8 total, 8 up, 8 in 2024-02-10T06:54:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:53 smithi184 ceph-mon[33513]: osdmap e475: 8 total, 8 up, 8 in 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[29878]: osdmap e474: 8 total, 8 up, 8 in 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[29878]: osdmap e475: 8 total, 8 up, 8 in 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[39523]: osdmap e474: 8 total, 8 up, 8 in 2024-02-10T06:54:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:54:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:54:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:53 smithi174 ceph-mon[39523]: osdmap e475: 8 total, 8 up, 8 in 2024-02-10T06:54:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:54 smithi174 ceph-mon[29878]: pgmap v710: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:54:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:54 smithi174 ceph-mon[39523]: pgmap v710: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:54:54.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:54 smithi184 ceph-mon[33513]: pgmap v710: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[29878]: osdmap e476: 8 total, 8 up, 8 in 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-87"}]: dispatch 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-87"}]: dispatch 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[39523]: osdmap e476: 8 total, 8 up, 8 in 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-87"}]: dispatch 2024-02-10T06:54:55.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-87"}]: dispatch 2024-02-10T06:54:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:55 smithi184 ceph-mon[33513]: osdmap e476: 8 total, 8 up, 8 in 2024-02-10T06:54:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:54:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-87"}]: dispatch 2024-02-10T06:54:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-87"}]: dispatch 2024-02-10T06:54:56.431 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:56 smithi184 ceph-mon[33513]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:56.431 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:56 smithi184 ceph-mon[33513]: osdmap e477: 8 total, 8 up, 8 in 2024-02-10T06:54:56.431 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:56 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:56 smithi174 ceph-mon[29878]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:56 smithi174 ceph-mon[29878]: osdmap e477: 8 total, 8 up, 8 in 2024-02-10T06:54:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:56 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:56 smithi174 ceph-mon[39523]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:54:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:56 smithi174 ceph-mon[39523]: osdmap e477: 8 total, 8 up, 8 in 2024-02-10T06:54:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:56 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:54:56.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:54:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[29878]: osdmap e478: 8 total, 8 up, 8 in 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[39523]: osdmap e478: 8 total, 8 up, 8 in 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:57.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:57 smithi184 ceph-mon[33513]: osdmap e478: 8 total, 8 up, 8 in 2024-02-10T06:54:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:54:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:54:58.159 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:54:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:54:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:54:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[29878]: pgmap v716: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[29878]: osdmap e479: 8 total, 8 up, 8 in 2024-02-10T06:54:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[39523]: pgmap v716: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[39523]: osdmap e479: 8 total, 8 up, 8 in 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:58.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:58 smithi184 ceph-mon[33513]: pgmap v716: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:54:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:54:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:58 smithi184 ceph-mon[33513]: osdmap e479: 8 total, 8 up, 8 in 2024-02-10T06:54:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:54:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:54:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:59 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:54:59 smithi174 ceph-mon[29878]: osdmap e480: 8 total, 8 up, 8 in 2024-02-10T06:54:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:59 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:54:59 smithi174 ceph-mon[39523]: osdmap e480: 8 total, 8 up, 8 in 2024-02-10T06:54:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:59 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:54:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:54:59 smithi184 ceph-mon[33513]: osdmap e480: 8 total, 8 up, 8 in 2024-02-10T06:55:00.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:00 smithi174 ceph-mon[29878]: pgmap v719: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:55:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:00 smithi174 ceph-mon[29878]: osdmap e481: 8 total, 8 up, 8 in 2024-02-10T06:55:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:00 smithi174 ceph-mon[39523]: pgmap v719: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:55:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:00 smithi174 ceph-mon[39523]: osdmap e481: 8 total, 8 up, 8 in 2024-02-10T06:55:00.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:00 smithi184 ceph-mon[33513]: pgmap v719: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:55:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:00 smithi184 ceph-mon[33513]: osdmap e481: 8 total, 8 up, 8 in 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[29878]: osdmap e482: 8 total, 8 up, 8 in 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-90"}]: dispatch 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-90"}]: dispatch 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[39523]: osdmap e482: 8 total, 8 up, 8 in 2024-02-10T06:55:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-90"}]: dispatch 2024-02-10T06:55:01.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-90"}]: dispatch 2024-02-10T06:55:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:01 smithi184 ceph-mon[33513]: osdmap e482: 8 total, 8 up, 8 in 2024-02-10T06:55:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-90"}]: dispatch 2024-02-10T06:55:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-90"}]: dispatch 2024-02-10T06:55:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:02 smithi174 ceph-mon[29878]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-02-10T06:55:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:02 smithi174 ceph-mon[29878]: osdmap e483: 8 total, 8 up, 8 in 2024-02-10T06:55:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:02 smithi174 ceph-mon[39523]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-02-10T06:55:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:02 smithi174 ceph-mon[39523]: osdmap e483: 8 total, 8 up, 8 in 2024-02-10T06:55:02.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:02 smithi184 ceph-mon[33513]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-02-10T06:55:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:02 smithi184 ceph-mon[33513]: osdmap e483: 8 total, 8 up, 8 in 2024-02-10T06:55:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[29878]: osdmap e484: 8 total, 8 up, 8 in 2024-02-10T06:55:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[29878]: osdmap e485: 8 total, 8 up, 8 in 2024-02-10T06:55:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[39523]: osdmap e484: 8 total, 8 up, 8 in 2024-02-10T06:55:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[39523]: osdmap e485: 8 total, 8 up, 8 in 2024-02-10T06:55:03.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:03 smithi184 ceph-mon[33513]: osdmap e484: 8 total, 8 up, 8 in 2024-02-10T06:55:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:03 smithi184 ceph-mon[33513]: osdmap e485: 8 total, 8 up, 8 in 2024-02-10T06:55:03.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:04 smithi174 ceph-mon[29878]: pgmap v725: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:55:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:04 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:04 smithi174 ceph-mon[29878]: osdmap e486: 8 total, 8 up, 8 in 2024-02-10T06:55:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:04 smithi174 ceph-mon[39523]: pgmap v725: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:55:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:04 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:04 smithi174 ceph-mon[39523]: osdmap e486: 8 total, 8 up, 8 in 2024-02-10T06:55:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:04 smithi184 ceph-mon[33513]: pgmap v725: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:55:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:04 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:04 smithi184 ceph-mon[33513]: osdmap e486: 8 total, 8 up, 8 in 2024-02-10T06:55:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[29878]: pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[29878]: osdmap e487: 8 total, 8 up, 8 in 2024-02-10T06:55:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-92"}]: dispatch 2024-02-10T06:55:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-92"}]: dispatch 2024-02-10T06:55:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[39523]: pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[39523]: osdmap e487: 8 total, 8 up, 8 in 2024-02-10T06:55:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-92"}]: dispatch 2024-02-10T06:55:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-92"}]: dispatch 2024-02-10T06:55:06.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:55:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-mon[33513]: pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-mon[33513]: osdmap e487: 8 total, 8 up, 8 in 2024-02-10T06:55:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-92"}]: dispatch 2024-02-10T06:55:06.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-92"}]: dispatch 2024-02-10T06:55:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-mon[29878]: osdmap e488: 8 total, 8 up, 8 in 2024-02-10T06:55:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-mon[39523]: osdmap e488: 8 total, 8 up, 8 in 2024-02-10T06:55:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:07 smithi184 ceph-mon[33513]: osdmap e488: 8 total, 8 up, 8 in 2024-02-10T06:55:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:08.229 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:55:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:55:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[29878]: pgmap v731: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[29878]: osdmap e489: 8 total, 8 up, 8 in 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[39523]: pgmap v731: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[39523]: osdmap e489: 8 total, 8 up, 8 in 2024-02-10T06:55:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:08.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:08 smithi184 ceph-mon[33513]: pgmap v731: 196 pgs: 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:55:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:08 smithi184 ceph-mon[33513]: osdmap e489: 8 total, 8 up, 8 in 2024-02-10T06:55:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:08.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:09 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:09 smithi174 ceph-mon[29878]: osdmap e490: 8 total, 8 up, 8 in 2024-02-10T06:55:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:09 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:09 smithi174 ceph-mon[39523]: osdmap e490: 8 total, 8 up, 8 in 2024-02-10T06:55:09.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:09.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:09 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:09 smithi184 ceph-mon[33513]: osdmap e490: 8 total, 8 up, 8 in 2024-02-10T06:55:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:10 smithi184 ceph-mon[33513]: pgmap v734: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:55:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:10 smithi184 ceph-mon[33513]: osdmap e491: 8 total, 8 up, 8 in 2024-02-10T06:55:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:10 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:10 smithi174 ceph-mon[29878]: pgmap v734: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:55:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:10 smithi174 ceph-mon[29878]: osdmap e491: 8 total, 8 up, 8 in 2024-02-10T06:55:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:10 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:10 smithi174 ceph-mon[39523]: pgmap v734: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T06:55:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:10 smithi174 ceph-mon[39523]: osdmap e491: 8 total, 8 up, 8 in 2024-02-10T06:55:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:10 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:11 smithi184 ceph-mon[33513]: osdmap e492: 8 total, 8 up, 8 in 2024-02-10T06:55:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:11 smithi174 ceph-mon[29878]: osdmap e492: 8 total, 8 up, 8 in 2024-02-10T06:55:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:11 smithi174 ceph-mon[39523]: osdmap e492: 8 total, 8 up, 8 in 2024-02-10T06:55:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:12 smithi184 ceph-mon[33513]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T06:55:12.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:12 smithi174 ceph-mon[29878]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T06:55:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:12 smithi174 ceph-mon[39523]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T06:55:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:14 smithi184 ceph-mon[33513]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-10T06:55:14.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:14 smithi174 ceph-mon[29878]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-10T06:55:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:14 smithi174 ceph-mon[39523]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-10T06:55:16.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:55:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:55:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:16 smithi174 ceph-mon[29878]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T06:55:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:16 smithi174 ceph-mon[39523]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T06:55:17.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:16 smithi184 ceph-mon[33513]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T06:55:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:55:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:55:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:55:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:18.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:19.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:18 smithi174 ceph-mon[29878]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T06:55:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:18 smithi174 ceph-mon[39523]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T06:55:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:18 smithi184 ceph-mon[33513]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T06:55:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:20 smithi174 ceph-mon[29878]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 832 B/s wr, 3 op/s 2024-02-10T06:55:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:20 smithi174 ceph-mon[39523]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 832 B/s wr, 3 op/s 2024-02-10T06:55:21.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:20 smithi184 ceph-mon[33513]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 832 B/s wr, 3 op/s 2024-02-10T06:55:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:21 smithi174 ceph-mon[29878]: osdmap e493: 8 total, 8 up, 8 in 2024-02-10T06:55:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:21 smithi174 ceph-mon[29878]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:21 smithi174 ceph-mon[39523]: osdmap e493: 8 total, 8 up, 8 in 2024-02-10T06:55:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:21 smithi174 ceph-mon[39523]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:22.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:21 smithi184 ceph-mon[33513]: osdmap e493: 8 total, 8 up, 8 in 2024-02-10T06:55:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:21 smithi184 ceph-mon[33513]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:24 smithi174 ceph-mon[29878]: pgmap v744: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:24 smithi174 ceph-mon[39523]: pgmap v744: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:24.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:24 smithi184 ceph-mon[33513]: pgmap v744: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:26.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:55:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:55:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:26 smithi174 ceph-mon[29878]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:26 smithi174 ceph-mon[39523]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:27.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:26 smithi184 ceph-mon[33513]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T06:55:28.140 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:28.140 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:55:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:55:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:55:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:29.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:28 smithi174 ceph-mon[29878]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:29.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:28 smithi174 ceph-mon[39523]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:29.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:28 smithi184 ceph-mon[33513]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:30 smithi174 ceph-mon[29878]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:31.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:30 smithi174 ceph-mon[29878]: osdmap e494: 8 total, 8 up, 8 in 2024-02-10T06:55:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:30 smithi174 ceph-mon[39523]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:31.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:30 smithi174 ceph-mon[39523]: osdmap e494: 8 total, 8 up, 8 in 2024-02-10T06:55:31.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:30 smithi184 ceph-mon[33513]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:31.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:30 smithi184 ceph-mon[33513]: osdmap e494: 8 total, 8 up, 8 in 2024-02-10T06:55:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-94"}]: dispatch 2024-02-10T06:55:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-94"}]: dispatch 2024-02-10T06:55:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:32.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-94"}]: dispatch 2024-02-10T06:55:32.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-94"}]: dispatch 2024-02-10T06:55:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-94"}]: dispatch 2024-02-10T06:55:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-94"}]: dispatch 2024-02-10T06:55:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:32 smithi174 ceph-mon[29878]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-10T06:55:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:32 smithi174 ceph-mon[29878]: osdmap e495: 8 total, 8 up, 8 in 2024-02-10T06:55:33.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:32 smithi174 ceph-mon[39523]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-10T06:55:33.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:32 smithi174 ceph-mon[39523]: osdmap e495: 8 total, 8 up, 8 in 2024-02-10T06:55:33.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:32 smithi184 ceph-mon[33513]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-10T06:55:33.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:32 smithi184 ceph-mon[33513]: osdmap e495: 8 total, 8 up, 8 in 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[29878]: osdmap e496: 8 total, 8 up, 8 in 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[29878]: pgmap v752: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[39523]: osdmap e496: 8 total, 8 up, 8 in 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:33 smithi174 ceph-mon[39523]: pgmap v752: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:33 smithi184 ceph-mon[33513]: osdmap e496: 8 total, 8 up, 8 in 2024-02-10T06:55:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:33 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:33 smithi184 ceph-mon[33513]: pgmap v752: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[29878]: osdmap e497: 8 total, 8 up, 8 in 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:35.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[39523]: osdmap e497: 8 total, 8 up, 8 in 2024-02-10T06:55:35.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:35.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:34 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:34 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:34 smithi184 ceph-mon[33513]: osdmap e497: 8 total, 8 up, 8 in 2024-02-10T06:55:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:55:36.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:35 smithi174 ceph-mon[29878]: osdmap e498: 8 total, 8 up, 8 in 2024-02-10T06:55:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:35 smithi174 ceph-mon[29878]: pgmap v755: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:36.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:35 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:35 smithi174 ceph-mon[39523]: osdmap e498: 8 total, 8 up, 8 in 2024-02-10T06:55:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:35 smithi174 ceph-mon[39523]: pgmap v755: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:35 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:35 smithi184 ceph-mon[33513]: osdmap e498: 8 total, 8 up, 8 in 2024-02-10T06:55:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:35 smithi184 ceph-mon[33513]: pgmap v755: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:35 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:55:36.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:55:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi184 to 3248M 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: Adjusting osd_memory_target on smithi174 to 3248M 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:37.241 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:37.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi184 to 3248M 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: Adjusting osd_memory_target on smithi174 to 3248M 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.242 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:37.243 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi184 to 3248M 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: Adjusting osd_memory_target on smithi174 to 3248M 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:37.374 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:37 smithi174 ceph-mon[29878]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 990 B/s rd, 990 B/s wr, 3 op/s 2024-02-10T06:55:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:55:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:55:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:55:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:37 smithi174 ceph-mon[39523]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 990 B/s rd, 990 B/s wr, 3 op/s 2024-02-10T06:55:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:37 smithi184 ceph-mon[33513]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 990 B/s rd, 990 B/s wr, 3 op/s 2024-02-10T06:55:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:40 smithi174 ceph-mon[29878]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 831 B/s rd, 831 B/s wr, 2 op/s 2024-02-10T06:55:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:40 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:40 smithi174 ceph-mon[39523]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 831 B/s rd, 831 B/s wr, 2 op/s 2024-02-10T06:55:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:40 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:40.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:40 smithi184 ceph-mon[33513]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 831 B/s rd, 831 B/s wr, 2 op/s 2024-02-10T06:55:40.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:40 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:42 smithi174 ceph-mon[29878]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-10T06:55:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:42 smithi174 ceph-mon[39523]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-10T06:55:42.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:42 smithi184 ceph-mon[33513]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-10T06:55:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:44 smithi174 ceph-mon[29878]: pgmap v759: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 559 B/s wr, 2 op/s 2024-02-10T06:55:44.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:44 smithi174 ceph-mon[39523]: pgmap v759: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 559 B/s wr, 2 op/s 2024-02-10T06:55:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:44 smithi184 ceph-mon[33513]: pgmap v759: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 559 B/s wr, 2 op/s 2024-02-10T06:55:45.239 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 06:55:44 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T06:55:44.960+0000 7fac621c9700 -1 snap_mapper.add_oid found existing snaps mapped on 112:0ac62037:test-rados-api-smithi174-79953-97::foo:21, removing 2024-02-10T06:55:45.239 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 06:55:44 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1[50156]: 2024-02-10T06:55:44.960+0000 7f4c59219700 -1 snap_mapper.add_oid found existing snaps mapped on 112:0ac62037:test-rados-api-smithi174-79953-97::foo:21, removing 2024-02-10T06:55:45.372 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 06:55:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T06:55:44.961+0000 7f8783485700 -1 snap_mapper.add_oid found existing snaps mapped on 112:0ac62037:test-rados-api-smithi174-79953-97::foo:21, removing 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-96"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-96"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-96"}]: dispatch 2024-02-10T06:55:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:45 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-96"}]: dispatch 2024-02-10T06:55:46.240 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:46.240 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:46.240 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-96"}]: dispatch 2024-02-10T06:55:46.240 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:45 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-96"}]: dispatch 2024-02-10T06:55:46.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:55:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:55:46.623 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 06:55:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=cleanup t=2024-02-10T06:55:46.239583571Z level=info msg="Completed cleanup jobs" duration=2.21552ms 2024-02-10T06:55:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:46 smithi174 ceph-mon[29878]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1008 B/s rd, 504 B/s wr, 2 op/s 2024-02-10T06:55:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:46 smithi174 ceph-mon[29878]: osdmap e499: 8 total, 8 up, 8 in 2024-02-10T06:55:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:46 smithi174 ceph-mon[39523]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1008 B/s rd, 504 B/s wr, 2 op/s 2024-02-10T06:55:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:46 smithi174 ceph-mon[39523]: osdmap e499: 8 total, 8 up, 8 in 2024-02-10T06:55:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:46 smithi184 ceph-mon[33513]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1008 B/s rd, 504 B/s wr, 2 op/s 2024-02-10T06:55:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:46 smithi184 ceph-mon[33513]: osdmap e499: 8 total, 8 up, 8 in 2024-02-10T06:55:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:55:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:55:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[29878]: osdmap e500: 8 total, 8 up, 8 in 2024-02-10T06:55:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[29878]: pgmap v763: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-10T06:55:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[39523]: osdmap e500: 8 total, 8 up, 8 in 2024-02-10T06:55:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:47 smithi174 ceph-mon[39523]: pgmap v763: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-10T06:55:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:47 smithi184 ceph-mon[33513]: osdmap e500: 8 total, 8 up, 8 in 2024-02-10T06:55:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:55:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:47 smithi184 ceph-mon[33513]: pgmap v763: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[29878]: osdmap e501: 8 total, 8 up, 8 in 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:49.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:49.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[39523]: osdmap e501: 8 total, 8 up, 8 in 2024-02-10T06:55:49.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:55:49.241 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:48 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:55:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:48 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:55:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:48 smithi184 ceph-mon[33513]: osdmap e501: 8 total, 8 up, 8 in 2024-02-10T06:55:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:55:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:48 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:55:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:55:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[29878]: osdmap e502: 8 total, 8 up, 8 in 2024-02-10T06:55:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]: dispatch 2024-02-10T06:55:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]: dispatch 2024-02-10T06:55:50.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[29878]: pgmap v766: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-10T06:55:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:55:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[39523]: osdmap e502: 8 total, 8 up, 8 in 2024-02-10T06:55:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]: dispatch 2024-02-10T06:55:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]: dispatch 2024-02-10T06:55:50.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:49 smithi174 ceph-mon[39523]: pgmap v766: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-10T06:55:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:55:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:49 smithi184 ceph-mon[33513]: osdmap e502: 8 total, 8 up, 8 in 2024-02-10T06:55:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]: dispatch 2024-02-10T06:55:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:49 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]: dispatch 2024-02-10T06:55:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:49 smithi184 ceph-mon[33513]: pgmap v766: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]': finished 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[29878]: osdmap e503: 8 total, 8 up, 8 in 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]': finished 2024-02-10T06:55:51.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[39523]: osdmap e503: 8 total, 8 up, 8 in 2024-02-10T06:55:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:51.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:50 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:51.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:50 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:55:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-98", "mode": "writeback"}]': finished 2024-02-10T06:55:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:50 smithi184 ceph-mon[33513]: osdmap e503: 8 total, 8 up, 8 in 2024-02-10T06:55:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:50 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]': finished 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: osdmap e504: 8 total, 8 up, 8 in 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:52.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]': finished 2024-02-10T06:55:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: osdmap e504: 8 total, 8 up, 8 in 2024-02-10T06:55:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:55:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:55:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:55:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:52.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:51 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:52.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-98"}]': finished 2024-02-10T06:55:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: osdmap e504: 8 total, 8 up, 8 in 2024-02-10T06:55:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:55:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:55:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:55:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:55:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:51 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[29878]: osdmap e505: 8 total, 8 up, 8 in 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:55:53.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[39523]: osdmap e505: 8 total, 8 up, 8 in 2024-02-10T06:55:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T06:55:53.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T06:55:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:52 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:55:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:55:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:52 smithi184 ceph-mon[33513]: osdmap e505: 8 total, 8 up, 8 in 2024-02-10T06:55:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T06:55:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T06:55:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]': finished 2024-02-10T06:55:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[29878]: osdmap e506: 8 total, 8 up, 8 in 2024-02-10T06:55:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:55:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:55:54.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[29878]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:55:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]': finished 2024-02-10T06:55:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[39523]: osdmap e506: 8 total, 8 up, 8 in 2024-02-10T06:55:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:55:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:55:54.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:53 smithi174 ceph-mon[39523]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:55:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_count","val": "1"}]': finished 2024-02-10T06:55:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:53 smithi184 ceph-mon[33513]: osdmap e506: 8 total, 8 up, 8 in 2024-02-10T06:55:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:55:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:55:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:53 smithi184 ceph-mon[33513]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[29878]: osdmap e507: 8 total, 8 up, 8 in 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[39523]: osdmap e507: 8 total, 8 up, 8 in 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T06:55:55.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:54 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:55:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:54 smithi184 ceph-mon[33513]: osdmap e507: 8 total, 8 up, 8 in 2024-02-10T06:55:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T06:55:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T06:55:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:54 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:55:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]': finished 2024-02-10T06:55:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[29878]: osdmap e508: 8 total, 8 up, 8 in 2024-02-10T06:55:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:56.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[29878]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:56.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]': finished 2024-02-10T06:55:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[39523]: osdmap e508: 8 total, 8 up, 8 in 2024-02-10T06:55:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:56.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:55 smithi174 ceph-mon[39523]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:56.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-98","var": "target_max_objects","val": "250"}]': finished 2024-02-10T06:55:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:55 smithi184 ceph-mon[33513]: osdmap e508: 8 total, 8 up, 8 in 2024-02-10T06:55:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:55:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:55 smithi184 ceph-mon[33513]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:55:56.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:55:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:55:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:55:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:57 smithi184 ceph-mon[33513]: osdmap e509: 8 total, 8 up, 8 in 2024-02-10T06:55:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[29878]: osdmap e509: 8 total, 8 up, 8 in 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[39523]: osdmap e509: 8 total, 8 up, 8 in 2024-02-10T06:55:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]: dispatch 2024-02-10T06:55:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:55:58.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:58 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]': finished 2024-02-10T06:55:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:58 smithi174 ceph-mon[29878]: osdmap e510: 8 total, 8 up, 8 in 2024-02-10T06:55:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:58 smithi174 ceph-mon[29878]: pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:55:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:55:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:55:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:58 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]': finished 2024-02-10T06:55:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:58 smithi174 ceph-mon[39523]: osdmap e510: 8 total, 8 up, 8 in 2024-02-10T06:55:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:58 smithi174 ceph-mon[39523]: pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:58 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-98"}]': finished 2024-02-10T06:55:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:58 smithi184 ceph-mon[33513]: osdmap e510: 8 total, 8 up, 8 in 2024-02-10T06:55:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:58 smithi184 ceph-mon[33513]: pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:55:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:55:59 smithi184 ceph-mon[33513]: osdmap e511: 8 total, 8 up, 8 in 2024-02-10T06:55:59.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:55:59 smithi174 ceph-mon[29878]: osdmap e511: 8 total, 8 up, 8 in 2024-02-10T06:55:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:55:59 smithi174 ceph-mon[39523]: osdmap e511: 8 total, 8 up, 8 in 2024-02-10T06:56:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:00 smithi184 ceph-mon[33513]: osdmap e512: 8 total, 8 up, 8 in 2024-02-10T06:56:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:00 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:00 smithi184 ceph-mon[33513]: pgmap v781: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[29878]: osdmap e512: 8 total, 8 up, 8 in 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[29878]: pgmap v781: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[39523]: osdmap e512: 8 total, 8 up, 8 in 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:00 smithi174 ceph-mon[39523]: pgmap v781: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:01 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:01 smithi184 ceph-mon[33513]: osdmap e513: 8 total, 8 up, 8 in 2024-02-10T06:56:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:01 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[39523]: osdmap e513: 8 total, 8 up, 8 in 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[29878]: osdmap e513: 8 total, 8 up, 8 in 2024-02-10T06:56:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:01.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:01 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:56:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:02 smithi184 ceph-mon[33513]: osdmap e514: 8 total, 8 up, 8 in 2024-02-10T06:56:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:02 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:02 smithi184 ceph-mon[33513]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[29878]: osdmap e514: 8 total, 8 up, 8 in 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[29878]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[39523]: osdmap e514: 8 total, 8 up, 8 in 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:02.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:02 smithi174 ceph-mon[39523]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:03.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]': finished 2024-02-10T06:56:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:03 smithi184 ceph-mon[33513]: osdmap e515: 8 total, 8 up, 8 in 2024-02-10T06:56:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]: dispatch 2024-02-10T06:56:03.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:03 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]: dispatch 2024-02-10T06:56:03.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]': finished 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[29878]: osdmap e515: 8 total, 8 up, 8 in 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]: dispatch 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]: dispatch 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-6", "overlaypool": "test-rados-api-smithi174-79953-100"}]': finished 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[39523]: osdmap e515: 8 total, 8 up, 8 in 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]: dispatch 2024-02-10T06:56:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:03 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]: dispatch 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]': finished 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: osdmap e516: 8 total, 8 up, 8 in 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:04 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]': finished 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: osdmap e516: 8 total, 8 up, 8 in 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-100", "mode": "writeback"}]': finished 2024-02-10T06:56:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: osdmap e516: 8 total, 8 up, 8 in 2024-02-10T06:56:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:04.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:04 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: osdmap e517: 8 total, 8 up, 8 in 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: osdmap e518: 8 total, 8 up, 8 in 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: osdmap e517: 8 total, 8 up, 8 in 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: osdmap e518: 8 total, 8 up, 8 in 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: osdmap e517: 8 total, 8 up, 8 in 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:05.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:56:05.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: osdmap e518: 8 total, 8 up, 8 in 2024-02-10T06:56:05.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:05.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-mon[33513]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:56:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-mon[33513]: osdmap e519: 8 total, 8 up, 8 in 2024-02-10T06:56:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T06:56:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[29878]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[29878]: osdmap e519: 8 total, 8 up, 8 in 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[39523]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:56:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[39523]: osdmap e519: 8 total, 8 up, 8 in 2024-02-10T06:56:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T06:56:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T06:56:06.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:56:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:56:07.805 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: osdmap e520: 8 total, 8 up, 8 in 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:07.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T06:56:07.807 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: osdmap e520: 8 total, 8 up, 8 in 2024-02-10T06:56:07.807 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:07.807 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:56:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T06:56:07.875 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: osdmap e520: 8 total, 8 up, 8 in 2024-02-10T06:56:07.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:07.876 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:08.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:56:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:56:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:56:08.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:08 smithi184 ceph-mon[33513]: pgmap v793: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:56:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:08 smithi184 ceph-mon[33513]: osdmap e521: 8 total, 8 up, 8 in 2024-02-10T06:56:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[29878]: pgmap v793: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[29878]: osdmap e521: 8 total, 8 up, 8 in 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[39523]: pgmap v793: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:08.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]': finished 2024-02-10T06:56:08.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[39523]: osdmap e521: 8 total, 8 up, 8 in 2024-02-10T06:56:08.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:08.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]: dispatch 2024-02-10T06:56:10.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:10 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]': finished 2024-02-10T06:56:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:10 smithi184 ceph-mon[33513]: osdmap e522: 8 total, 8 up, 8 in 2024-02-10T06:56:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:10 smithi184 ceph-mon[33513]: pgmap v796: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:10 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:10.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]': finished 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[29878]: osdmap e522: 8 total, 8 up, 8 in 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[29878]: pgmap v796: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-100"}]': finished 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[39523]: osdmap e522: 8 total, 8 up, 8 in 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[39523]: pgmap v796: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:10 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:11.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:11 smithi184 ceph-mon[33513]: osdmap e523: 8 total, 8 up, 8 in 2024-02-10T06:56:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:11 smithi184 ceph-mon[33513]: osdmap e524: 8 total, 8 up, 8 in 2024-02-10T06:56:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:11 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[29878]: osdmap e523: 8 total, 8 up, 8 in 2024-02-10T06:56:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[29878]: osdmap e524: 8 total, 8 up, 8 in 2024-02-10T06:56:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[39523]: osdmap e523: 8 total, 8 up, 8 in 2024-02-10T06:56:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[39523]: osdmap e524: 8 total, 8 up, 8 in 2024-02-10T06:56:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:11 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:12 smithi184 ceph-mon[33513]: pgmap v799: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:12 smithi184 ceph-mon[33513]: osdmap e525: 8 total, 8 up, 8 in 2024-02-10T06:56:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:56:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:56:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:56:12.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[29878]: pgmap v799: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[29878]: osdmap e525: 8 total, 8 up, 8 in 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[39523]: pgmap v799: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[39523]: osdmap e525: 8 total, 8 up, 8 in 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T06:56:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:56:12.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T06:56:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:56:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:14 smithi184 ceph-mon[33513]: osdmap e526: 8 total, 8 up, 8 in 2024-02-10T06:56:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:56:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:56:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:14 smithi184 ceph-mon[33513]: pgmap v802: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[29878]: osdmap e526: 8 total, 8 up, 8 in 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[29878]: pgmap v802: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[39523]: osdmap e526: 8 total, 8 up, 8 in 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T06:56:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:14 smithi174 ceph-mon[39523]: pgmap v802: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:56:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:15 smithi184 ceph-mon[33513]: osdmap e527: 8 total, 8 up, 8 in 2024-02-10T06:56:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:56:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:15 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[29878]: osdmap e527: 8 total, 8 up, 8 in 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[39523]: osdmap e527: 8 total, 8 up, 8 in 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:56:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:15 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: osdmap e528: 8 total, 8 up, 8 in 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: pgmap v805: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-102"}]: dispatch 2024-02-10T06:56:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-102"}]: dispatch 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: osdmap e528: 8 total, 8 up, 8 in 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: pgmap v805: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-102"}]: dispatch 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-102"}]: dispatch 2024-02-10T06:56:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T06:56:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: osdmap e528: 8 total, 8 up, 8 in 2024-02-10T06:56:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: pgmap v805: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-6"}]: dispatch 2024-02-10T06:56:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3851220089' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-102"}]: dispatch 2024-02-10T06:56:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:16 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-6", "tierpool": "test-rados-api-smithi174-79953-102"}]: dispatch 2024-02-10T06:56:16.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:56:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:56:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:17 smithi184 ceph-mon[33513]: osdmap e529: 8 total, 8 up, 8 in 2024-02-10T06:56:17.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:17.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:17 smithi174 ceph-mon[29878]: osdmap e529: 8 total, 8 up, 8 in 2024-02-10T06:56:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:17 smithi174 ceph-mon[39523]: osdmap e529: 8 total, 8 up, 8 in 2024-02-10T06:56:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:18.110 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:56:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:56:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: pgmap v807: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 254 B/s wr, 2 op/s 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: osdmap e530: 8 total, 8 up, 8 in 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:18 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: pgmap v807: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 254 B/s wr, 2 op/s 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: osdmap e530: 8 total, 8 up, 8 in 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: pgmap v807: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 254 B/s wr, 2 op/s 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: osdmap e530: 8 total, 8 up, 8 in 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:18 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:19 smithi184 ceph-mon[33513]: osdmap e531: 8 total, 8 up, 8 in 2024-02-10T06:56:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:19 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[29878]: osdmap e531: 8 total, 8 up, 8 in 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[39523]: osdmap e531: 8 total, 8 up, 8 in 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:19 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:20.123 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:20 smithi174 ceph-mon[29878]: pgmap v810: 164 pgs: 164 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:20.123 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:20 smithi174 ceph-mon[29878]: osdmap e532: 8 total, 8 up, 8 in 2024-02-10T06:56:20.124 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:20 smithi174 ceph-mon[39523]: pgmap v810: 164 pgs: 164 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:20.124 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:20 smithi174 ceph-mon[39523]: osdmap e532: 8 total, 8 up, 8 in 2024-02-10T06:56:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:20 smithi184 ceph-mon[33513]: pgmap v810: 164 pgs: 164 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:20 smithi184 ceph-mon[33513]: osdmap e532: 8 total, 8 up, 8 in 2024-02-10T06:56:21.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:21 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:21 smithi174 ceph-mon[29878]: osdmap e533: 8 total, 8 up, 8 in 2024-02-10T06:56:21.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:21 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:21.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:21 smithi174 ceph-mon[39523]: osdmap e533: 8 total, 8 up, 8 in 2024-02-10T06:56:21.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:21 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi174-79953-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:21.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:21 smithi184 ceph-mon[33513]: osdmap e533: 8 total, 8 up, 8 in 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[29878]: pgmap v813: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[29878]: osdmap e534: 8 total, 8 up, 8 in 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[39523]: pgmap v813: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:22.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[39523]: osdmap e534: 8 total, 8 up, 8 in 2024-02-10T06:56:22.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:22 smithi184 ceph-mon[33513]: pgmap v813: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:22 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:22 smithi184 ceph-mon[33513]: osdmap e534: 8 total, 8 up, 8 in 2024-02-10T06:56:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:23 smithi174 ceph-mon[29878]: osdmap e535: 8 total, 8 up, 8 in 2024-02-10T06:56:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:23 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:23 smithi174 ceph-mon[39523]: osdmap e535: 8 total, 8 up, 8 in 2024-02-10T06:56:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:23 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:23 smithi184 ceph-mon[33513]: osdmap e535: 8 total, 8 up, 8 in 2024-02-10T06:56:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:23 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:24 smithi174 ceph-mon[29878]: pgmap v816: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:24 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:24 smithi174 ceph-mon[29878]: osdmap e536: 8 total, 8 up, 8 in 2024-02-10T06:56:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:24 smithi174 ceph-mon[39523]: pgmap v816: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:24 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:24 smithi174 ceph-mon[39523]: osdmap e536: 8 total, 8 up, 8 in 2024-02-10T06:56:24.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:24 smithi184 ceph-mon[33513]: pgmap v816: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:24 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:24 smithi184 ceph-mon[33513]: osdmap e536: 8 total, 8 up, 8 in 2024-02-10T06:56:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:25 smithi174 ceph-mon[29878]: osdmap e537: 8 total, 8 up, 8 in 2024-02-10T06:56:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:25 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:25 smithi174 ceph-mon[39523]: osdmap e537: 8 total, 8 up, 8 in 2024-02-10T06:56:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:25 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:25.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:25 smithi184 ceph-mon[33513]: osdmap e537: 8 total, 8 up, 8 in 2024-02-10T06:56:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:25 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[29878]: pgmap v819: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[29878]: osdmap e538: 8 total, 8 up, 8 in 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[39523]: pgmap v819: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[39523]: osdmap e538: 8 total, 8 up, 8 in 2024-02-10T06:56:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:26.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:26 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:26.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:26 smithi184 ceph-mon[33513]: pgmap v819: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:26.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:26.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:26 smithi184 ceph-mon[33513]: osdmap e538: 8 total, 8 up, 8 in 2024-02-10T06:56:26.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:26.514 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:26 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:26.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:56:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:56:27.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[29878]: osdmap e539: 8 total, 8 up, 8 in 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[39523]: osdmap e539: 8 total, 8 up, 8 in 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:27.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:27 smithi184 ceph-mon[33513]: osdmap e539: 8 total, 8 up, 8 in 2024-02-10T06:56:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:27 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:27 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:27.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:28.190 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:56:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:56:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:56:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[29878]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[29878]: osdmap e540: 8 total, 8 up, 8 in 2024-02-10T06:56:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[39523]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[39523]: osdmap e540: 8 total, 8 up, 8 in 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:28.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:28 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:28 smithi184 ceph-mon[33513]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:28 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-107", "overlaypool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:28.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:28 smithi184 ceph-mon[33513]: osdmap e540: 8 total, 8 up, 8 in 2024-02-10T06:56:28.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:28.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:28 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]': finished 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[29878]: osdmap e541: 8 total, 8 up, 8 in 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]: dispatch 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]: dispatch 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]': finished 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[39523]: osdmap e541: 8 total, 8 up, 8 in 2024-02-10T06:56:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]: dispatch 2024-02-10T06:56:29.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:29 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]: dispatch 2024-02-10T06:56:29.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:29 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:29.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-107-cache", "mode": "writeback"}]': finished 2024-02-10T06:56:29.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:29 smithi184 ceph-mon[33513]: osdmap e541: 8 total, 8 up, 8 in 2024-02-10T06:56:29.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]: dispatch 2024-02-10T06:56:29.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:29 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]: dispatch 2024-02-10T06:56:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[29878]: pgmap v825: 236 pgs: 236 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]': finished 2024-02-10T06:56:30.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[29878]: osdmap e542: 8 total, 8 up, 8 in 2024-02-10T06:56:30.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:30.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:30.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[39523]: pgmap v825: 236 pgs: 236 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]': finished 2024-02-10T06:56:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[39523]: osdmap e542: 8 total, 8 up, 8 in 2024-02-10T06:56:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:30.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:30 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:30 smithi184 ceph-mon[33513]: pgmap v825: 236 pgs: 236 active+clean; 582 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T06:56:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-107"}]': finished 2024-02-10T06:56:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:30 smithi184 ceph-mon[33513]: osdmap e542: 8 total, 8 up, 8 in 2024-02-10T06:56:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/4187646801' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:30 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]: dispatch 2024-02-10T06:56:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:31 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:31 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:31 smithi174 ceph-mon[39523]: osdmap e543: 8 total, 8 up, 8 in 2024-02-10T06:56:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:31 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:31 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:31.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:31 smithi174 ceph-mon[29878]: osdmap e543: 8 total, 8 up, 8 in 2024-02-10T06:56:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:31 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:31 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-107", "tierpool": "test-rados-api-smithi174-79953-107-cache"}]': finished 2024-02-10T06:56:31.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:31 smithi184 ceph-mon[33513]: osdmap e543: 8 total, 8 up, 8 in 2024-02-10T06:56:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:32 smithi174 ceph-mon[39523]: pgmap v828: 236 pgs: 236 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:56:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:32 smithi174 ceph-mon[39523]: osdmap e544: 8 total, 8 up, 8 in 2024-02-10T06:56:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:32 smithi174 ceph-mon[29878]: pgmap v828: 236 pgs: 236 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:56:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:32 smithi174 ceph-mon[29878]: osdmap e544: 8 total, 8 up, 8 in 2024-02-10T06:56:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:32 smithi184 ceph-mon[33513]: pgmap v828: 236 pgs: 236 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:56:32.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:32 smithi184 ceph-mon[33513]: osdmap e544: 8 total, 8 up, 8 in 2024-02-10T06:56:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[29878]: osdmap e545: 8 total, 8 up, 8 in 2024-02-10T06:56:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[39523]: osdmap e545: 8 total, 8 up, 8 in 2024-02-10T06:56:33.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:33.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:33.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:33.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:33 smithi184 ceph-mon[33513]: osdmap e545: 8 total, 8 up, 8 in 2024-02-10T06:56:33.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:33.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:33.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:34.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[29878]: pgmap v831: 172 pgs: 172 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[29878]: osdmap e546: 8 total, 8 up, 8 in 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[39523]: pgmap v831: 172 pgs: 172 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[39523]: osdmap e546: 8 total, 8 up, 8 in 2024-02-10T06:56:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:34.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:34 smithi184 ceph-mon[33513]: pgmap v831: 172 pgs: 172 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:34.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:34.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:34 smithi184 ceph-mon[33513]: osdmap e546: 8 total, 8 up, 8 in 2024-02-10T06:56:34.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:35.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:35 smithi174 ceph-mon[39523]: osdmap e547: 8 total, 8 up, 8 in 2024-02-10T06:56:35.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:35.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:35 smithi174 ceph-mon[39523]: osdmap e548: 8 total, 8 up, 8 in 2024-02-10T06:56:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:35 smithi174 ceph-mon[29878]: osdmap e547: 8 total, 8 up, 8 in 2024-02-10T06:56:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:35 smithi174 ceph-mon[29878]: osdmap e548: 8 total, 8 up, 8 in 2024-02-10T06:56:35.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:35 smithi184 ceph-mon[33513]: osdmap e547: 8 total, 8 up, 8 in 2024-02-10T06:56:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:35.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:35 smithi184 ceph-mon[33513]: osdmap e548: 8 total, 8 up, 8 in 2024-02-10T06:56:36.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:56:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:56:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[29878]: pgmap v834: 172 pgs: 172 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[29878]: osdmap e549: 8 total, 8 up, 8 in 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[39523]: pgmap v834: 172 pgs: 172 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[39523]: osdmap e549: 8 total, 8 up, 8 in 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:37.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:36 smithi184 ceph-mon[33513]: pgmap v834: 172 pgs: 172 active+clean; 582 KiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:36 smithi184 ceph-mon[33513]: osdmap e549: 8 total, 8 up, 8 in 2024-02-10T06:56:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:56:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:56:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: osdmap e550: 8 total, 8 up, 8 in 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:56:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: osdmap e550: 8 total, 8 up, 8 in 2024-02-10T06:56:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:56:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:56:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:56:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: osdmap e550: 8 total, 8 up, 8 in 2024-02-10T06:56:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[29878]: pgmap v837: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[29878]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[29878]: osdmap e551: 8 total, 8 up, 8 in 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-109", "overlaypool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[39523]: pgmap v837: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[39523]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:39.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[39523]: osdmap e551: 8 total, 8 up, 8 in 2024-02-10T06:56:39.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-109", "overlaypool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:39.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:38 smithi184 ceph-mon[33513]: pgmap v837: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:38 smithi184 ceph-mon[33513]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:38 smithi184 ceph-mon[33513]: osdmap e551: 8 total, 8 up, 8 in 2024-02-10T06:56:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-109", "overlaypool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:40 smithi184 ceph-mon[33513]: pgmap v840: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-109", "overlaypool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:40 smithi184 ceph-mon[33513]: osdmap e552: 8 total, 8 up, 8 in 2024-02-10T06:56:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-109-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[29878]: pgmap v840: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-109", "overlaypool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[29878]: osdmap e552: 8 total, 8 up, 8 in 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-109-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[39523]: pgmap v840: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-109", "overlaypool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[39523]: osdmap e552: 8 total, 8 up, 8 in 2024-02-10T06:56:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-109-cache", "mode": "writeback"}]: dispatch 2024-02-10T06:56:41.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:41 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-109-cache", "mode": "writeback"}]': finished 2024-02-10T06:56:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:41 smithi184 ceph-mon[33513]: osdmap e553: 8 total, 8 up, 8 in 2024-02-10T06:56:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:56:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-109-cache", "mode": "writeback"}]': finished 2024-02-10T06:56:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[29878]: osdmap e553: 8 total, 8 up, 8 in 2024-02-10T06:56:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:56:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:56:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-109-cache", "mode": "writeback"}]': finished 2024-02-10T06:56:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[39523]: osdmap e553: 8 total, 8 up, 8 in 2024-02-10T06:56:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:56:41.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:56:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:42 smithi184 ceph-mon[33513]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:42 smithi184 ceph-mon[33513]: osdmap e554: 8 total, 8 up, 8 in 2024-02-10T06:56:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:56:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:42 smithi184 ceph-mon[33513]: osdmap e555: 8 total, 8 up, 8 in 2024-02-10T06:56:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:42.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[29878]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[29878]: osdmap e554: 8 total, 8 up, 8 in 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[29878]: osdmap e555: 8 total, 8 up, 8 in 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[39523]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[39523]: osdmap e554: 8 total, 8 up, 8 in 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:56:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:56:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[39523]: osdmap e555: 8 total, 8 up, 8 in 2024-02-10T06:56:42.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:56:43.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:43 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:56:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:43 smithi184 ceph-mon[33513]: osdmap e556: 8 total, 8 up, 8 in 2024-02-10T06:56:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[29878]: osdmap e556: 8 total, 8 up, 8 in 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[39523]: osdmap e556: 8 total, 8 up, 8 in 2024-02-10T06:56:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T06:56:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:44 smithi184 ceph-mon[33513]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T06:56:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:44 smithi184 ceph-mon[33513]: osdmap e557: 8 total, 8 up, 8 in 2024-02-10T06:56:44.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:44 smithi174 ceph-mon[29878]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T06:56:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:44 smithi174 ceph-mon[29878]: osdmap e557: 8 total, 8 up, 8 in 2024-02-10T06:56:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:44 smithi174 ceph-mon[39523]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T06:56:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:44 smithi174 ceph-mon[39523]: osdmap e557: 8 total, 8 up, 8 in 2024-02-10T06:56:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:45.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:45 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:45.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:45 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:45.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:45.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:45 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:46.535 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:46 smithi184 ceph-mon[33513]: pgmap v849: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:46.535 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:46.535 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:46 smithi184 ceph-mon[33513]: osdmap e558: 8 total, 8 up, 8 in 2024-02-10T06:56:46.535 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:46.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[29878]: pgmap v849: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[29878]: osdmap e558: 8 total, 8 up, 8 in 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[39523]: pgmap v849: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[39523]: osdmap e558: 8 total, 8 up, 8 in 2024-02-10T06:56:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]: dispatch 2024-02-10T06:56:46.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:56:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:56:47.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:47 smithi184 ceph-mon[33513]: osdmap e559: 8 total, 8 up, 8 in 2024-02-10T06:56:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-mon[29878]: osdmap e559: 8 total, 8 up, 8 in 2024-02-10T06:56:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-109", "tierpool": "test-rados-api-smithi174-79953-109-cache"}]': finished 2024-02-10T06:56:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-mon[39523]: osdmap e559: 8 total, 8 up, 8 in 2024-02-10T06:56:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:48.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:56:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:56:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:56:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:48 smithi184 ceph-mon[33513]: pgmap v852: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:56:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:48 smithi184 ceph-mon[33513]: osdmap e560: 8 total, 8 up, 8 in 2024-02-10T06:56:48.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:48 smithi174 ceph-mon[29878]: pgmap v852: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:56:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:48 smithi174 ceph-mon[29878]: osdmap e560: 8 total, 8 up, 8 in 2024-02-10T06:56:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:48 smithi174 ceph-mon[39523]: pgmap v852: 212 pgs: 212 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:56:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:48 smithi174 ceph-mon[39523]: osdmap e560: 8 total, 8 up, 8 in 2024-02-10T06:56:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:49 smithi184 ceph-mon[33513]: osdmap e561: 8 total, 8 up, 8 in 2024-02-10T06:56:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:49 smithi184 ceph-mon[33513]: osdmap e562: 8 total, 8 up, 8 in 2024-02-10T06:56:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[29878]: osdmap e561: 8 total, 8 up, 8 in 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[29878]: osdmap e562: 8 total, 8 up, 8 in 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[39523]: osdmap e561: 8 total, 8 up, 8 in 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[39523]: osdmap e562: 8 total, 8 up, 8 in 2024-02-10T06:56:49.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]: dispatch 2024-02-10T06:56:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:50 smithi184 ceph-mon[33513]: pgmap v855: 172 pgs: 172 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:50 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:50 smithi184 ceph-mon[33513]: osdmap e563: 8 total, 8 up, 8 in 2024-02-10T06:56:50.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[29878]: pgmap v855: 172 pgs: 172 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[29878]: osdmap e563: 8 total, 8 up, 8 in 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[39523]: pgmap v855: 172 pgs: 172 active+clean; 582 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2792917889' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-109"}]': finished 2024-02-10T06:56:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:50 smithi174 ceph-mon[39523]: osdmap e563: 8 total, 8 up, 8 in 2024-02-10T06:56:52.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:52 smithi184 ceph-mon[33513]: pgmap v858: 172 pgs: 172 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:56:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:52 smithi184 ceph-mon[33513]: osdmap e564: 8 total, 8 up, 8 in 2024-02-10T06:56:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:52 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[29878]: pgmap v858: 172 pgs: 172 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[29878]: osdmap e564: 8 total, 8 up, 8 in 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[39523]: pgmap v858: 172 pgs: 172 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[39523]: osdmap e564: 8 total, 8 up, 8 in 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:52.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:56:53.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:53 smithi184 ceph-mon[33513]: osdmap e565: 8 total, 8 up, 8 in 2024-02-10T06:56:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:53.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:53 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:53.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[29878]: osdmap e565: 8 total, 8 up, 8 in 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[39523]: osdmap e565: 8 total, 8 up, 8 in 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1394328171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:53 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]: dispatch 2024-02-10T06:56:54.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:54 smithi184 ceph-mon[33513]: pgmap v861: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:56:54.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:54 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:54.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:54 smithi184 ceph-mon[33513]: osdmap e566: 8 total, 8 up, 8 in 2024-02-10T06:56:54.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:54.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:54.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[29878]: pgmap v861: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[29878]: osdmap e566: 8 total, 8 up, 8 in 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[39523]: pgmap v861: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi174-79953-104"}]': finished 2024-02-10T06:56:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[39523]: osdmap e566: 8 total, 8 up, 8 in 2024-02-10T06:56:54.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:54.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:54.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T06:56:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:55 smithi184 ceph-mon[33513]: osdmap e567: 8 total, 8 up, 8 in 2024-02-10T06:56:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:55.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:55 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[29878]: osdmap e567: 8 total, 8 up, 8 in 2024-02-10T06:56:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:55.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi174-79953-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T06:56:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[39523]: osdmap e567: 8 total, 8 up, 8 in 2024-02-10T06:56:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:56:55.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:55 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:56:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:56 smithi184 ceph-mon[33513]: pgmap v864: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:56.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:56 smithi184 ceph-mon[33513]: osdmap e568: 8 total, 8 up, 8 in 2024-02-10T06:56:56.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:56:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:56:56.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:56 smithi174 ceph-mon[29878]: pgmap v864: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:56.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:56 smithi174 ceph-mon[29878]: osdmap e568: 8 total, 8 up, 8 in 2024-02-10T06:56:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:56 smithi174 ceph-mon[39523]: pgmap v864: 164 pgs: 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:56:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:56 smithi174 ceph-mon[39523]: osdmap e568: 8 total, 8 up, 8 in 2024-02-10T06:56:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:56:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:57 smithi184 ceph-mon[33513]: osdmap e569: 8 total, 8 up, 8 in 2024-02-10T06:56:57.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:56:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-mon[29878]: osdmap e569: 8 total, 8 up, 8 in 2024-02-10T06:56:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi174-79953-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:56:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-mon[39523]: osdmap e569: 8 total, 8 up, 8 in 2024-02-10T06:56:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:56:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:56:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:56:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:56:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:58 smithi184 ceph-mon[33513]: pgmap v867: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:58 smithi184 ceph-mon[33513]: osdmap e570: 8 total, 8 up, 8 in 2024-02-10T06:56:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:58 smithi174 ceph-mon[29878]: pgmap v867: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:58 smithi174 ceph-mon[29878]: osdmap e570: 8 total, 8 up, 8 in 2024-02-10T06:56:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:58 smithi174 ceph-mon[39523]: pgmap v867: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:56:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:58 smithi174 ceph-mon[39523]: osdmap e570: 8 total, 8 up, 8 in 2024-02-10T06:56:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:56:59.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:59 smithi184 ceph-mon[33513]: osdmap e571: 8 total, 8 up, 8 in 2024-02-10T06:56:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:56:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:59.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:59.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:59 smithi174 ceph-mon[29878]: osdmap e571: 8 total, 8 up, 8 in 2024-02-10T06:56:59.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:56:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:56:59.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:56:59.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:59 smithi174 ceph-mon[39523]: osdmap e571: 8 total, 8 up, 8 in 2024-02-10T06:56:59.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:56:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:00 smithi184 ceph-mon[33513]: pgmap v870: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:00 smithi184 ceph-mon[33513]: osdmap e572: 8 total, 8 up, 8 in 2024-02-10T06:57:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-112"}]: dispatch 2024-02-10T06:57:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:00 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[29878]: pgmap v870: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[29878]: osdmap e572: 8 total, 8 up, 8 in 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-112"}]: dispatch 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[39523]: pgmap v870: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[39523]: osdmap e572: 8 total, 8 up, 8 in 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-112"}]: dispatch 2024-02-10T06:57:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:00 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-112"}]': finished 2024-02-10T06:57:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:01 smithi184 ceph-mon[33513]: osdmap e573: 8 total, 8 up, 8 in 2024-02-10T06:57:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:01.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-112"}]': finished 2024-02-10T06:57:01.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:01 smithi174 ceph-mon[29878]: osdmap e573: 8 total, 8 up, 8 in 2024-02-10T06:57:01.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:01.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-112"}]': finished 2024-02-10T06:57:01.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:01 smithi174 ceph-mon[39523]: osdmap e573: 8 total, 8 up, 8 in 2024-02-10T06:57:01.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[29878]: pgmap v873: 204 pgs: 204 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[29878]: osdmap e574: 8 total, 8 up, 8 in 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112"}]: dispatch 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[39523]: pgmap v873: 204 pgs: 204 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:02.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[39523]: osdmap e574: 8 total, 8 up, 8 in 2024-02-10T06:57:02.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112"}]: dispatch 2024-02-10T06:57:02.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:02 smithi184 ceph-mon[33513]: pgmap v873: 204 pgs: 204 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:57:02.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:02.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:02 smithi184 ceph-mon[33513]: osdmap e574: 8 total, 8 up, 8 in 2024-02-10T06:57:02.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112"}]: dispatch 2024-02-10T06:57:03.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112"}]': finished 2024-02-10T06:57:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:03 smithi174 ceph-mon[29878]: osdmap e575: 8 total, 8 up, 8 in 2024-02-10T06:57:03.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112"}]': finished 2024-02-10T06:57:03.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:03 smithi174 ceph-mon[39523]: osdmap e575: 8 total, 8 up, 8 in 2024-02-10T06:57:03.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-112"}]': finished 2024-02-10T06:57:03.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:03 smithi184 ceph-mon[33513]: osdmap e575: 8 total, 8 up, 8 in 2024-02-10T06:57:04.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:04 smithi174 ceph-mon[29878]: pgmap v876: 204 pgs: 204 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:57:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:04 smithi174 ceph-mon[29878]: osdmap e576: 8 total, 8 up, 8 in 2024-02-10T06:57:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:04 smithi174 ceph-mon[39523]: pgmap v876: 204 pgs: 204 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:57:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:04 smithi174 ceph-mon[39523]: osdmap e576: 8 total, 8 up, 8 in 2024-02-10T06:57:04.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:04 smithi184 ceph-mon[33513]: pgmap v876: 204 pgs: 204 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:57:04.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:04 smithi184 ceph-mon[33513]: osdmap e576: 8 total, 8 up, 8 in 2024-02-10T06:57:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:05 smithi174 ceph-mon[29878]: osdmap e577: 8 total, 8 up, 8 in 2024-02-10T06:57:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:05 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:05 smithi174 ceph-mon[39523]: osdmap e577: 8 total, 8 up, 8 in 2024-02-10T06:57:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:05 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:05.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:05 smithi184 ceph-mon[33513]: osdmap e577: 8 total, 8 up, 8 in 2024-02-10T06:57:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:05 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[29878]: pgmap v879: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[29878]: osdmap e578: 8 total, 8 up, 8 in 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[39523]: pgmap v879: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[39523]: osdmap e578: 8 total, 8 up, 8 in 2024-02-10T06:57:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:06.774 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:06 smithi184 ceph-mon[33513]: pgmap v879: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:06.774 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:06.774 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:06 smithi184 ceph-mon[33513]: osdmap e578: 8 total, 8 up, 8 in 2024-02-10T06:57:06.774 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:06.775 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:57:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: osdmap e579: 8 total, 8 up, 8 in 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-114"}]: dispatch 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-114"}]': finished 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: osdmap e580: 8 total, 8 up, 8 in 2024-02-10T06:57:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-114", "mode": "writeback"}]: dispatch 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: osdmap e579: 8 total, 8 up, 8 in 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-114"}]: dispatch 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-114"}]': finished 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: osdmap e580: 8 total, 8 up, 8 in 2024-02-10T06:57:07.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-114", "mode": "writeback"}]: dispatch 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: osdmap e579: 8 total, 8 up, 8 in 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-114"}]: dispatch 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-114"}]': finished 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: osdmap e580: 8 total, 8 up, 8 in 2024-02-10T06:57:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-114", "mode": "writeback"}]: dispatch 2024-02-10T06:57:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:57:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:57:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[39523]: pgmap v882: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:57:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-114", "mode": "writeback"}]': finished 2024-02-10T06:57:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[39523]: osdmap e581: 8 total, 8 up, 8 in 2024-02-10T06:57:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[29878]: pgmap v882: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:57:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:08.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-114", "mode": "writeback"}]': finished 2024-02-10T06:57:08.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:08 smithi174 ceph-mon[29878]: osdmap e581: 8 total, 8 up, 8 in 2024-02-10T06:57:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:08 smithi184 ceph-mon[33513]: pgmap v882: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T06:57:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:08 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-114", "mode": "writeback"}]': finished 2024-02-10T06:57:08.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:08 smithi184 ceph-mon[33513]: osdmap e581: 8 total, 8 up, 8 in 2024-02-10T06:57:09.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:09.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:09.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[29878]: pgmap v885: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[29878]: osdmap e582: 8 total, 8 up, 8 in 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114"}]: dispatch 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[39523]: pgmap v885: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[39523]: osdmap e582: 8 total, 8 up, 8 in 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114"}]: dispatch 2024-02-10T06:57:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:10 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:10 smithi184 ceph-mon[33513]: pgmap v885: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:57:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:10 smithi184 ceph-mon[33513]: osdmap e582: 8 total, 8 up, 8 in 2024-02-10T06:57:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114"}]: dispatch 2024-02-10T06:57:10.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:10 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:11.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:11 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114"}]': finished 2024-02-10T06:57:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:11 smithi174 ceph-mon[29878]: osdmap e583: 8 total, 8 up, 8 in 2024-02-10T06:57:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:11 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114"}]': finished 2024-02-10T06:57:11.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:11 smithi174 ceph-mon[39523]: osdmap e583: 8 total, 8 up, 8 in 2024-02-10T06:57:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:11 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-114"}]': finished 2024-02-10T06:57:11.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:11 smithi184 ceph-mon[33513]: osdmap e583: 8 total, 8 up, 8 in 2024-02-10T06:57:12.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:12 smithi174 ceph-mon[29878]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:57:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:12 smithi174 ceph-mon[29878]: osdmap e584: 8 total, 8 up, 8 in 2024-02-10T06:57:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:12 smithi174 ceph-mon[39523]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:57:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:12 smithi174 ceph-mon[39523]: osdmap e584: 8 total, 8 up, 8 in 2024-02-10T06:57:12.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:12 smithi184 ceph-mon[33513]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T06:57:12.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:12 smithi184 ceph-mon[33513]: osdmap e584: 8 total, 8 up, 8 in 2024-02-10T06:57:13.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:13 smithi174 ceph-mon[29878]: osdmap e585: 8 total, 8 up, 8 in 2024-02-10T06:57:13.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:13 smithi174 ceph-mon[39523]: osdmap e585: 8 total, 8 up, 8 in 2024-02-10T06:57:13.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:13.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:13 smithi184 ceph-mon[33513]: osdmap e585: 8 total, 8 up, 8 in 2024-02-10T06:57:13.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:14 smithi174 ceph-mon[29878]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:14 smithi174 ceph-mon[29878]: osdmap e586: 8 total, 8 up, 8 in 2024-02-10T06:57:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:14 smithi174 ceph-mon[39523]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:14 smithi174 ceph-mon[39523]: osdmap e586: 8 total, 8 up, 8 in 2024-02-10T06:57:14.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:14 smithi184 ceph-mon[33513]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:14.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:14 smithi184 ceph-mon[33513]: osdmap e586: 8 total, 8 up, 8 in 2024-02-10T06:57:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:15 smithi174 ceph-mon[29878]: osdmap e587: 8 total, 8 up, 8 in 2024-02-10T06:57:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:15 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:15 smithi174 ceph-mon[39523]: osdmap e587: 8 total, 8 up, 8 in 2024-02-10T06:57:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:15 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:15.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:15 smithi184 ceph-mon[33513]: osdmap e587: 8 total, 8 up, 8 in 2024-02-10T06:57:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:15.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:15 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:16.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:16 smithi184 ceph-mon[33513]: pgmap v894: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:16.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:16.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:16 smithi184 ceph-mon[33513]: osdmap e588: 8 total, 8 up, 8 in 2024-02-10T06:57:16.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-116"}]: dispatch 2024-02-10T06:57:16.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:57:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[29878]: pgmap v894: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[29878]: osdmap e588: 8 total, 8 up, 8 in 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-116"}]: dispatch 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[39523]: pgmap v894: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:16.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[39523]: osdmap e588: 8 total, 8 up, 8 in 2024-02-10T06:57:16.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-116"}]: dispatch 2024-02-10T06:57:17.805 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-116"}]': finished 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[29878]: osdmap e589: 8 total, 8 up, 8 in 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-116", "mode": "writeback"}]: dispatch 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-116", "mode": "writeback"}]': finished 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-116"}]': finished 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[39523]: osdmap e589: 8 total, 8 up, 8 in 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-116", "mode": "writeback"}]: dispatch 2024-02-10T06:57:17.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:17.807 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:17.807 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-116", "mode": "writeback"}]': finished 2024-02-10T06:57:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-116"}]': finished 2024-02-10T06:57:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:17 smithi184 ceph-mon[33513]: osdmap e589: 8 total, 8 up, 8 in 2024-02-10T06:57:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-116", "mode": "writeback"}]: dispatch 2024-02-10T06:57:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:17 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:17.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-116", "mode": "writeback"}]': finished 2024-02-10T06:57:18.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:57:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:57:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:18.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:18 smithi184 ceph-mon[33513]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:57:18.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:18 smithi184 ceph-mon[33513]: osdmap e590: 8 total, 8 up, 8 in 2024-02-10T06:57:18.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-10T06:57:18.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:18 smithi184 ceph-mon[33513]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[29878]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[29878]: osdmap e590: 8 total, 8 up, 8 in 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[29878]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[39523]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[39523]: osdmap e590: 8 total, 8 up, 8 in 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-10T06:57:18.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:18 smithi174 ceph-mon[39523]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.7"}]: dispatch 2024-02-10T06:57:19.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:19 smithi184 ceph-mon[33513]: 326.7 scrub starts 2024-02-10T06:57:19.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:19 smithi184 ceph-mon[33513]: 326.7 scrub ok 2024-02-10T06:57:19.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:19 smithi174 ceph-mon[29878]: 326.7 scrub starts 2024-02-10T06:57:19.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:19 smithi174 ceph-mon[29878]: 326.7 scrub ok 2024-02-10T06:57:19.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:19 smithi174 ceph-mon[39523]: 326.7 scrub starts 2024-02-10T06:57:19.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:19 smithi174 ceph-mon[39523]: 326.7 scrub ok 2024-02-10T06:57:20.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:20 smithi184 ceph-mon[33513]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:57:20.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:20 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:20 smithi174 ceph-mon[29878]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:57:20.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:20 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:20.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:20 smithi174 ceph-mon[39523]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T06:57:20.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:20 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:22 smithi174 ceph-mon[29878]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:57:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:22.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:22 smithi174 ceph-mon[39523]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:57:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:22.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:23.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:22 smithi184 ceph-mon[33513]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T06:57:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:24.988 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:24 smithi174 ceph-mon[29878]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 939 B/s wr, 2 op/s 2024-02-10T06:57:24.989 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:24 smithi174 ceph-mon[39523]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 939 B/s wr, 2 op/s 2024-02-10T06:57:25.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:24 smithi184 ceph-mon[33513]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 939 B/s wr, 2 op/s 2024-02-10T06:57:26.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:25 smithi174 ceph-mon[29878]: pgmap v902: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 711 B/s rd, 0 op/s 2024-02-10T06:57:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:25 smithi174 ceph-mon[39523]: pgmap v902: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 711 B/s rd, 0 op/s 2024-02-10T06:57:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:25 smithi184 ceph-mon[33513]: pgmap v902: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 711 B/s rd, 0 op/s 2024-02-10T06:57:26.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:57:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:57:27.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:26 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:27.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:26 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:27.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:26 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:28.140 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:57:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:57:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:28.141 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:27 smithi174 ceph-mon[29878]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T06:57:28.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:27 smithi174 ceph-mon[39523]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T06:57:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:27 smithi184 ceph-mon[33513]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T06:57:30.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:30 smithi174 ceph-mon[29878]: pgmap v904: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 1 op/s 2024-02-10T06:57:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:30 smithi174 ceph-mon[39523]: pgmap v904: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 1 op/s 2024-02-10T06:57:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:30 smithi184 ceph-mon[33513]: pgmap v904: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 1 op/s 2024-02-10T06:57:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:32 smithi174 ceph-mon[29878]: pgmap v905: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:57:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:32 smithi174 ceph-mon[39523]: pgmap v905: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:57:32.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:32 smithi184 ceph-mon[33513]: pgmap v905: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:57:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:34 smithi174 ceph-mon[29878]: pgmap v906: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T06:57:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:34 smithi174 ceph-mon[29878]: osdmap e591: 8 total, 8 up, 8 in 2024-02-10T06:57:34.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:34 smithi174 ceph-mon[39523]: pgmap v906: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T06:57:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:34 smithi174 ceph-mon[39523]: osdmap e591: 8 total, 8 up, 8 in 2024-02-10T06:57:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:34.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:34 smithi184 ceph-mon[33513]: pgmap v906: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T06:57:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:34 smithi184 ceph-mon[33513]: osdmap e591: 8 total, 8 up, 8 in 2024-02-10T06:57:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[29878]: osdmap e592: 8 total, 8 up, 8 in 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116"}]: dispatch 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116"}]': finished 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[39523]: osdmap e592: 8 total, 8 up, 8 in 2024-02-10T06:57:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116"}]: dispatch 2024-02-10T06:57:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:35.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:35 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116"}]': finished 2024-02-10T06:57:35.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:35 smithi184 ceph-mon[33513]: osdmap e592: 8 total, 8 up, 8 in 2024-02-10T06:57:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116"}]: dispatch 2024-02-10T06:57:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:35 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:35.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:35 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-116"}]': finished 2024-02-10T06:57:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:36 smithi174 ceph-mon[29878]: pgmap v909: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:57:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:36 smithi174 ceph-mon[29878]: osdmap e593: 8 total, 8 up, 8 in 2024-02-10T06:57:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:36 smithi174 ceph-mon[39523]: pgmap v909: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:57:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:36 smithi174 ceph-mon[39523]: osdmap e593: 8 total, 8 up, 8 in 2024-02-10T06:57:36.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:36 smithi184 ceph-mon[33513]: pgmap v909: 204 pgs: 204 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T06:57:36.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:36 smithi184 ceph-mon[33513]: osdmap e593: 8 total, 8 up, 8 in 2024-02-10T06:57:36.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:57:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:57:37.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[29878]: osdmap e594: 8 total, 8 up, 8 in 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[39523]: osdmap e594: 8 total, 8 up, 8 in 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:37.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:37 smithi184 ceph-mon[33513]: osdmap e594: 8 total, 8 up, 8 in 2024-02-10T06:57:37.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:37.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:37.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:37.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:57:38.074 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:57:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:57:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[39523]: pgmap v912: 172 pgs: 172 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[39523]: osdmap e595: 8 total, 8 up, 8 in 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[39523]: osdmap e596: 8 total, 8 up, 8 in 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[29878]: pgmap v912: 172 pgs: 172 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[29878]: osdmap e595: 8 total, 8 up, 8 in 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:38.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:38 smithi174 ceph-mon[29878]: osdmap e596: 8 total, 8 up, 8 in 2024-02-10T06:57:38.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:38 smithi184 ceph-mon[33513]: pgmap v912: 172 pgs: 172 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T06:57:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:38 smithi184 ceph-mon[33513]: osdmap e595: 8 total, 8 up, 8 in 2024-02-10T06:57:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:38 smithi184 ceph-mon[33513]: osdmap e596: 8 total, 8 up, 8 in 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: pgmap v915: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: osdmap e597: 8 total, 8 up, 8 in 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: osdmap e598: 8 total, 8 up, 8 in 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-118"}]: dispatch 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: pgmap v915: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: osdmap e597: 8 total, 8 up, 8 in 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: osdmap e598: 8 total, 8 up, 8 in 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-118"}]: dispatch 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:40.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:40 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: pgmap v915: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: osdmap e597: 8 total, 8 up, 8 in 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: osdmap e598: 8 total, 8 up, 8 in 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-118"}]: dispatch 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:40 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-118"}]': finished 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[29878]: osdmap e599: 8 total, 8 up, 8 in 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-118", "mode": "writeback"}]: dispatch 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[29878]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-118"}]': finished 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[39523]: osdmap e599: 8 total, 8 up, 8 in 2024-02-10T06:57:42.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-118", "mode": "writeback"}]: dispatch 2024-02-10T06:57:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[39523]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-02-10T06:57:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:42.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:41 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:42.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-118"}]': finished 2024-02-10T06:57:42.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:41 smithi184 ceph-mon[33513]: osdmap e599: 8 total, 8 up, 8 in 2024-02-10T06:57:42.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-118", "mode": "writeback"}]: dispatch 2024-02-10T06:57:42.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:41 smithi184 ceph-mon[33513]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-02-10T06:57:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:41 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:41 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-118", "mode": "writeback"}]': finished 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[29878]: osdmap e600: 8 total, 8 up, 8 in 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:43.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-118", "mode": "writeback"}]': finished 2024-02-10T06:57:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[39523]: osdmap e600: 8 total, 8 up, 8 in 2024-02-10T06:57:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:57:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:57:43.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:42 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:42 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-118", "mode": "writeback"}]': finished 2024-02-10T06:57:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:42 smithi184 ceph-mon[33513]: osdmap e600: 8 total, 8 up, 8 in 2024-02-10T06:57:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:42 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:57:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:42 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:57:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:42 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:43 smithi174 ceph-mon[29878]: osdmap e601: 8 total, 8 up, 8 in 2024-02-10T06:57:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:44.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:43 smithi174 ceph-mon[29878]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-02-10T06:57:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:43 smithi174 ceph-mon[39523]: osdmap e601: 8 total, 8 up, 8 in 2024-02-10T06:57:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:44.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:43 smithi174 ceph-mon[39523]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-02-10T06:57:44.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:43 smithi184 ceph-mon[33513]: osdmap e601: 8 total, 8 up, 8 in 2024-02-10T06:57:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:43 smithi184 ceph-mon[33513]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-02-10T06:57:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:44 smithi174 ceph-mon[29878]: osdmap e602: 8 total, 8 up, 8 in 2024-02-10T06:57:45.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118"}]: dispatch 2024-02-10T06:57:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:44 smithi174 ceph-mon[39523]: osdmap e602: 8 total, 8 up, 8 in 2024-02-10T06:57:45.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118"}]: dispatch 2024-02-10T06:57:45.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:44 smithi184 ceph-mon[33513]: osdmap e602: 8 total, 8 up, 8 in 2024-02-10T06:57:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118"}]: dispatch 2024-02-10T06:57:46.238 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118"}]': finished 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[39523]: osdmap e603: 8 total, 8 up, 8 in 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[39523]: pgmap v925: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118"}]': finished 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[29878]: osdmap e603: 8 total, 8 up, 8 in 2024-02-10T06:57:46.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:45 smithi174 ceph-mon[29878]: pgmap v925: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:46.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:45 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:46.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-118"}]': finished 2024-02-10T06:57:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:45 smithi184 ceph-mon[33513]: osdmap e603: 8 total, 8 up, 8 in 2024-02-10T06:57:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:45 smithi184 ceph-mon[33513]: pgmap v925: 204 pgs: 204 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:46.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:57:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:57:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:46 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:46 smithi174 ceph-mon[29878]: osdmap e604: 8 total, 8 up, 8 in 2024-02-10T06:57:47.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:46 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:46 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:46 smithi174 ceph-mon[39523]: osdmap e604: 8 total, 8 up, 8 in 2024-02-10T06:57:47.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:46 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:46 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:46 smithi184 ceph-mon[33513]: osdmap e604: 8 total, 8 up, 8 in 2024-02-10T06:57:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:46 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-mon[29878]: osdmap e605: 8 total, 8 up, 8 in 2024-02-10T06:57:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-mon[29878]: pgmap v928: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:57:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-mon[39523]: osdmap e605: 8 total, 8 up, 8 in 2024-02-10T06:57:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:47 smithi174 ceph-mon[39523]: pgmap v928: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:47 smithi184 ceph-mon[33513]: osdmap e605: 8 total, 8 up, 8 in 2024-02-10T06:57:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:47 smithi184 ceph-mon[33513]: pgmap v928: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:49.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:49 smithi184 ceph-mon[33513]: osdmap e606: 8 total, 8 up, 8 in 2024-02-10T06:57:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:49 smithi174 ceph-mon[29878]: osdmap e606: 8 total, 8 up, 8 in 2024-02-10T06:57:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:49.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:49.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:49 smithi174 ceph-mon[39523]: osdmap e606: 8 total, 8 up, 8 in 2024-02-10T06:57:49.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:50.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: osdmap e607: 8 total, 8 up, 8 in 2024-02-10T06:57:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-120"}]: dispatch 2024-02-10T06:57:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: pgmap v931: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-120"}]': finished 2024-02-10T06:57:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: osdmap e608: 8 total, 8 up, 8 in 2024-02-10T06:57:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-120", "mode": "writeback"}]: dispatch 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: osdmap e607: 8 total, 8 up, 8 in 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-120"}]: dispatch 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: pgmap v931: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-120"}]': finished 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: osdmap e608: 8 total, 8 up, 8 in 2024-02-10T06:57:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-120", "mode": "writeback"}]: dispatch 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: osdmap e607: 8 total, 8 up, 8 in 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-120"}]: dispatch 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: pgmap v931: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-120"}]': finished 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: osdmap e608: 8 total, 8 up, 8 in 2024-02-10T06:57:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-120", "mode": "writeback"}]: dispatch 2024-02-10T06:57:51.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:51 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-120", "mode": "writeback"}]': finished 2024-02-10T06:57:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:51 smithi184 ceph-mon[33513]: osdmap e609: 8 total, 8 up, 8 in 2024-02-10T06:57:51.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:51 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:51.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-120", "mode": "writeback"}]': finished 2024-02-10T06:57:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:51 smithi174 ceph-mon[29878]: osdmap e609: 8 total, 8 up, 8 in 2024-02-10T06:57:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:51 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-120", "mode": "writeback"}]': finished 2024-02-10T06:57:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:51 smithi174 ceph-mon[39523]: osdmap e609: 8 total, 8 up, 8 in 2024-02-10T06:57:52.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:52 smithi184 ceph-mon[33513]: pgmap v934: 204 pgs: 204 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:57:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[29878]: pgmap v934: 204 pgs: 204 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[39523]: pgmap v934: 204 pgs: 204 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:57:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:57:53.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:53 smithi184 ceph-mon[33513]: osdmap e610: 8 total, 8 up, 8 in 2024-02-10T06:57:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120"}]: dispatch 2024-02-10T06:57:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:53 smithi174 ceph-mon[29878]: osdmap e610: 8 total, 8 up, 8 in 2024-02-10T06:57:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120"}]: dispatch 2024-02-10T06:57:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:57:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:53 smithi174 ceph-mon[39523]: osdmap e610: 8 total, 8 up, 8 in 2024-02-10T06:57:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120"}]: dispatch 2024-02-10T06:57:54.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:54 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120"}]': finished 2024-02-10T06:57:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:54 smithi184 ceph-mon[33513]: osdmap e611: 8 total, 8 up, 8 in 2024-02-10T06:57:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:54 smithi184 ceph-mon[33513]: pgmap v937: 204 pgs: 204 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:57:54.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120"}]': finished 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[29878]: osdmap e611: 8 total, 8 up, 8 in 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[29878]: pgmap v937: 204 pgs: 204 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-120"}]': finished 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[39523]: osdmap e611: 8 total, 8 up, 8 in 2024-02-10T06:57:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:54 smithi174 ceph-mon[39523]: pgmap v937: 204 pgs: 204 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T06:57:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:55 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:55 smithi184 ceph-mon[33513]: osdmap e612: 8 total, 8 up, 8 in 2024-02-10T06:57:55.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:55 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:55 smithi174 ceph-mon[29878]: osdmap e612: 8 total, 8 up, 8 in 2024-02-10T06:57:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:55 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:57:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:55 smithi174 ceph-mon[39523]: osdmap e612: 8 total, 8 up, 8 in 2024-02-10T06:57:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:56 smithi184 ceph-mon[33513]: osdmap e613: 8 total, 8 up, 8 in 2024-02-10T06:57:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:56 smithi184 ceph-mon[33513]: pgmap v940: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:56.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:56 smithi174 ceph-mon[29878]: osdmap e613: 8 total, 8 up, 8 in 2024-02-10T06:57:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:56 smithi174 ceph-mon[29878]: pgmap v940: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:56 smithi174 ceph-mon[39523]: osdmap e613: 8 total, 8 up, 8 in 2024-02-10T06:57:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:57:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:56 smithi174 ceph-mon[39523]: pgmap v940: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:57:56.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:57:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:57:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:57 smithi184 ceph-mon[33513]: osdmap e614: 8 total, 8 up, 8 in 2024-02-10T06:57:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[29878]: osdmap e614: 8 total, 8 up, 8 in 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[39523]: osdmap e614: 8 total, 8 up, 8 in 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:57:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:57:58.068 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:57:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:57:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:57:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:58 smithi184 ceph-mon[33513]: osdmap e615: 8 total, 8 up, 8 in 2024-02-10T06:57:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-122"}]: dispatch 2024-02-10T06:57:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:58 smithi184 ceph-mon[33513]: pgmap v943: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:57:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-122"}]': finished 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[39523]: osdmap e615: 8 total, 8 up, 8 in 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-122"}]: dispatch 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[39523]: pgmap v943: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-122"}]': finished 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[29878]: osdmap e615: 8 total, 8 up, 8 in 2024-02-10T06:57:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-122"}]: dispatch 2024-02-10T06:57:58.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[29878]: pgmap v943: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T06:57:58.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-122"}]': finished 2024-02-10T06:57:59.343 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[29878]: osdmap e616: 8 total, 8 up, 8 in 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-122", "mode": "writeback"}]: dispatch 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-122", "mode": "writeback"}]': finished 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[29878]: osdmap e617: 8 total, 8 up, 8 in 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[39523]: osdmap e616: 8 total, 8 up, 8 in 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-122", "mode": "writeback"}]: dispatch 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-122", "mode": "writeback"}]': finished 2024-02-10T06:57:59.344 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:57:59 smithi174 ceph-mon[39523]: osdmap e617: 8 total, 8 up, 8 in 2024-02-10T06:57:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:59 smithi184 ceph-mon[33513]: osdmap e616: 8 total, 8 up, 8 in 2024-02-10T06:57:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-122", "mode": "writeback"}]: dispatch 2024-02-10T06:57:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:59 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:57:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-122", "mode": "writeback"}]': finished 2024-02-10T06:57:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:57:59 smithi184 ceph-mon[33513]: osdmap e617: 8 total, 8 up, 8 in 2024-02-10T06:58:00.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:00 smithi184 ceph-mon[33513]: pgmap v946: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:00 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:00.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:00 smithi174 ceph-mon[29878]: pgmap v946: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:00 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:00 smithi174 ceph-mon[39523]: pgmap v946: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:00 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:01.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:01 smithi184 ceph-mon[33513]: osdmap e618: 8 total, 8 up, 8 in 2024-02-10T06:58:01.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122"}]: dispatch 2024-02-10T06:58:01.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:01 smithi174 ceph-mon[29878]: osdmap e618: 8 total, 8 up, 8 in 2024-02-10T06:58:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122"}]: dispatch 2024-02-10T06:58:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:01 smithi174 ceph-mon[39523]: osdmap e618: 8 total, 8 up, 8 in 2024-02-10T06:58:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122"}]: dispatch 2024-02-10T06:58:02.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:02 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:02 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122"}]': finished 2024-02-10T06:58:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:02 smithi184 ceph-mon[33513]: osdmap e619: 8 total, 8 up, 8 in 2024-02-10T06:58:02.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:02 smithi184 ceph-mon[33513]: pgmap v949: 204 pgs: 204 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-10T06:58:02.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:02.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122"}]': finished 2024-02-10T06:58:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[29878]: osdmap e619: 8 total, 8 up, 8 in 2024-02-10T06:58:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[29878]: pgmap v949: 204 pgs: 204 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-10T06:58:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-122"}]': finished 2024-02-10T06:58:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[39523]: osdmap e619: 8 total, 8 up, 8 in 2024-02-10T06:58:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:02 smithi174 ceph-mon[39523]: pgmap v949: 204 pgs: 204 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-10T06:58:03.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:03 smithi184 ceph-mon[33513]: osdmap e620: 8 total, 8 up, 8 in 2024-02-10T06:58:03.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:03 smithi174 ceph-mon[29878]: osdmap e620: 8 total, 8 up, 8 in 2024-02-10T06:58:03.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:03 smithi174 ceph-mon[39523]: osdmap e620: 8 total, 8 up, 8 in 2024-02-10T06:58:04.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:04 smithi184 ceph-mon[33513]: osdmap e621: 8 total, 8 up, 8 in 2024-02-10T06:58:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:04 smithi184 ceph-mon[33513]: pgmap v952: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:58:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:04 smithi174 ceph-mon[39523]: osdmap e621: 8 total, 8 up, 8 in 2024-02-10T06:58:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:04 smithi174 ceph-mon[39523]: pgmap v952: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:58:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:04 smithi174 ceph-mon[29878]: osdmap e621: 8 total, 8 up, 8 in 2024-02-10T06:58:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:04.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:04 smithi174 ceph-mon[29878]: pgmap v952: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T06:58:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:05 smithi174 ceph-mon[29878]: osdmap e622: 8 total, 8 up, 8 in 2024-02-10T06:58:05.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:05 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:05 smithi174 ceph-mon[39523]: osdmap e622: 8 total, 8 up, 8 in 2024-02-10T06:58:05.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:05 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:05.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:05 smithi184 ceph-mon[33513]: osdmap e622: 8 total, 8 up, 8 in 2024-02-10T06:58:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:05 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:06.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:06 smithi174 ceph-mon[29878]: osdmap e623: 8 total, 8 up, 8 in 2024-02-10T06:58:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:06 smithi174 ceph-mon[29878]: pgmap v955: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:06 smithi174 ceph-mon[39523]: osdmap e623: 8 total, 8 up, 8 in 2024-02-10T06:58:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:06 smithi174 ceph-mon[39523]: pgmap v955: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:06.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:06 smithi184 ceph-mon[33513]: osdmap e623: 8 total, 8 up, 8 in 2024-02-10T06:58:06.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:06 smithi184 ceph-mon[33513]: pgmap v955: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:06.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:06.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:58:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:58:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: osdmap e624: 8 total, 8 up, 8 in 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-124"}]: dispatch 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-124"}]': finished 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: osdmap e625: 8 total, 8 up, 8 in 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-124", "mode": "writeback"}]: dispatch 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: osdmap e624: 8 total, 8 up, 8 in 2024-02-10T06:58:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-124"}]: dispatch 2024-02-10T06:58:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-124"}]': finished 2024-02-10T06:58:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: osdmap e625: 8 total, 8 up, 8 in 2024-02-10T06:58:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-124", "mode": "writeback"}]: dispatch 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: osdmap e624: 8 total, 8 up, 8 in 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-124"}]: dispatch 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:07.545 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-124"}]': finished 2024-02-10T06:58:07.546 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: osdmap e625: 8 total, 8 up, 8 in 2024-02-10T06:58:07.546 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-124", "mode": "writeback"}]: dispatch 2024-02-10T06:58:08.149 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:58:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:58:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[29878]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-124", "mode": "writeback"}]': finished 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[29878]: osdmap e626: 8 total, 8 up, 8 in 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[39523]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-124", "mode": "writeback"}]': finished 2024-02-10T06:58:08.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:08 smithi174 ceph-mon[39523]: osdmap e626: 8 total, 8 up, 8 in 2024-02-10T06:58:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:08 smithi184 ceph-mon[33513]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:58:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:08 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-124", "mode": "writeback"}]': finished 2024-02-10T06:58:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:08 smithi184 ceph-mon[33513]: osdmap e626: 8 total, 8 up, 8 in 2024-02-10T06:58:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[29878]: pgmap v960: 204 pgs: 204 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[29878]: osdmap e627: 8 total, 8 up, 8 in 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[39523]: pgmap v960: 204 pgs: 204 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[39523]: osdmap e627: 8 total, 8 up, 8 in 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:10.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:10 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:10.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:10 smithi184 ceph-mon[33513]: pgmap v960: 204 pgs: 204 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T06:58:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:10 smithi184 ceph-mon[33513]: osdmap e627: 8 total, 8 up, 8 in 2024-02-10T06:58:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:10 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[29878]: osdmap e628: 8 total, 8 up, 8 in 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124"}]: dispatch 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124"}]': finished 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[39523]: osdmap e628: 8 total, 8 up, 8 in 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124"}]: dispatch 2024-02-10T06:58:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:11.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124"}]': finished 2024-02-10T06:58:11.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:11 smithi184 ceph-mon[33513]: osdmap e628: 8 total, 8 up, 8 in 2024-02-10T06:58:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124"}]: dispatch 2024-02-10T06:58:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:11 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-124"}]': finished 2024-02-10T06:58:12.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:12 smithi174 ceph-mon[29878]: pgmap v963: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T06:58:12.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:12 smithi174 ceph-mon[29878]: osdmap e629: 8 total, 8 up, 8 in 2024-02-10T06:58:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:12 smithi174 ceph-mon[39523]: pgmap v963: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T06:58:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:12 smithi174 ceph-mon[39523]: osdmap e629: 8 total, 8 up, 8 in 2024-02-10T06:58:12.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:12 smithi184 ceph-mon[33513]: pgmap v963: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T06:58:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:12 smithi184 ceph-mon[33513]: osdmap e629: 8 total, 8 up, 8 in 2024-02-10T06:58:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:13 smithi174 ceph-mon[29878]: osdmap e630: 8 total, 8 up, 8 in 2024-02-10T06:58:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:13 smithi174 ceph-mon[39523]: osdmap e630: 8 total, 8 up, 8 in 2024-02-10T06:58:13.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:13 smithi184 ceph-mon[33513]: osdmap e630: 8 total, 8 up, 8 in 2024-02-10T06:58:14.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:14 smithi174 ceph-mon[29878]: pgmap v966: 172 pgs: 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:14 smithi174 ceph-mon[29878]: osdmap e631: 8 total, 8 up, 8 in 2024-02-10T06:58:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:14 smithi174 ceph-mon[39523]: pgmap v966: 172 pgs: 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:14 smithi174 ceph-mon[39523]: osdmap e631: 8 total, 8 up, 8 in 2024-02-10T06:58:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:14.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:14 smithi184 ceph-mon[33513]: pgmap v966: 172 pgs: 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:14 smithi184 ceph-mon[33513]: osdmap e631: 8 total, 8 up, 8 in 2024-02-10T06:58:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[29878]: osdmap e632: 8 total, 8 up, 8 in 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[29878]: osdmap e633: 8 total, 8 up, 8 in 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-126"}]: dispatch 2024-02-10T06:58:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[39523]: osdmap e632: 8 total, 8 up, 8 in 2024-02-10T06:58:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[39523]: osdmap e633: 8 total, 8 up, 8 in 2024-02-10T06:58:15.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-126"}]: dispatch 2024-02-10T06:58:15.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:15 smithi184 ceph-mon[33513]: osdmap e632: 8 total, 8 up, 8 in 2024-02-10T06:58:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:15 smithi184 ceph-mon[33513]: osdmap e633: 8 total, 8 up, 8 in 2024-02-10T06:58:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-126"}]: dispatch 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[29878]: pgmap v970: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-126"}]': finished 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[29878]: osdmap e634: 8 total, 8 up, 8 in 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-126", "mode": "writeback"}]: dispatch 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[39523]: pgmap v970: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-126"}]': finished 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[39523]: osdmap e634: 8 total, 8 up, 8 in 2024-02-10T06:58:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-126", "mode": "writeback"}]: dispatch 2024-02-10T06:58:16.607 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:16 smithi184 ceph-mon[33513]: pgmap v970: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:16.607 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-126"}]': finished 2024-02-10T06:58:16.607 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:16 smithi184 ceph-mon[33513]: osdmap e634: 8 total, 8 up, 8 in 2024-02-10T06:58:16.607 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-126", "mode": "writeback"}]: dispatch 2024-02-10T06:58:16.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:58:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-126", "mode": "writeback"}]': finished 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[29878]: osdmap e635: 8 total, 8 up, 8 in 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-126", "mode": "writeback"}]': finished 2024-02-10T06:58:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[39523]: osdmap e635: 8 total, 8 up, 8 in 2024-02-10T06:58:17.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:17.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:17 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-126", "mode": "writeback"}]': finished 2024-02-10T06:58:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:17 smithi184 ceph-mon[33513]: osdmap e635: 8 total, 8 up, 8 in 2024-02-10T06:58:17.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:18.201 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:58:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:58:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[29878]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[29878]: osdmap e636: 8 total, 8 up, 8 in 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126"}]: dispatch 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[39523]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[39523]: osdmap e636: 8 total, 8 up, 8 in 2024-02-10T06:58:18.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126"}]: dispatch 2024-02-10T06:58:18.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:18 smithi184 ceph-mon[33513]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T06:58:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:18 smithi184 ceph-mon[33513]: osdmap e636: 8 total, 8 up, 8 in 2024-02-10T06:58:18.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126"}]: dispatch 2024-02-10T06:58:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:19 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126"}]': finished 2024-02-10T06:58:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:19 smithi174 ceph-mon[29878]: osdmap e637: 8 total, 8 up, 8 in 2024-02-10T06:58:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:19 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126"}]': finished 2024-02-10T06:58:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:19 smithi174 ceph-mon[39523]: osdmap e637: 8 total, 8 up, 8 in 2024-02-10T06:58:19.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:19 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:19.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-126"}]': finished 2024-02-10T06:58:19.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:19 smithi184 ceph-mon[33513]: osdmap e637: 8 total, 8 up, 8 in 2024-02-10T06:58:19.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:58:19.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9073 ms) 2024-02-10T06:58:19.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-10T06:58:19.978 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6042 ms) 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5022 ms) 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25718 ms) 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14005 ms) 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12086 ms) 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-10T06:58:19.979 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 00000012 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12050 ms) 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6040 ms) 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (558266 ms total) 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1221 ms) 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10867 ms) 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18071 ms) 2024-02-10T06:58:19.980 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30161 ms total) 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7060 ms) 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8067 ms) 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: waiting for scrub... 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: done waiting 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24748 ms) 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9775 ms) 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-10T06:58:19.981 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8066 ms) 2024-02-10T06:58:19.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-10T06:58:19.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8056 ms) 2024-02-10T06:58:19.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-10T06:58:19.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10084 ms) 2024-02-10T06:58:19.982 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-10T06:58:20.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:20 smithi174 ceph-mon[29878]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:20.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:20 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:20 smithi174 ceph-mon[29878]: osdmap e638: 8 total, 8 up, 8 in 2024-02-10T06:58:20.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:20 smithi174 ceph-mon[39523]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:20.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:20 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:20.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:20 smithi174 ceph-mon[39523]: osdmap e638: 8 total, 8 up, 8 in 2024-02-10T06:58:20.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:20 smithi184 ceph-mon[33513]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:20 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:20 smithi184 ceph-mon[33513]: osdmap e638: 8 total, 8 up, 8 in 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[29878]: osdmap e639: 8 total, 8 up, 8 in 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[29878]: pgmap v979: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[39523]: osdmap e639: 8 total, 8 up, 8 in 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[39523]: pgmap v979: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:22.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:21 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:22.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:21 smithi184 ceph-mon[33513]: osdmap e639: 8 total, 8 up, 8 in 2024-02-10T06:58:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:21 smithi184 ceph-mon[33513]: pgmap v979: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:21 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:22 smithi184 ceph-mon[33513]: osdmap e640: 8 total, 8 up, 8 in 2024-02-10T06:58:23.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:22 smithi174 ceph-mon[29878]: osdmap e640: 8 total, 8 up, 8 in 2024-02-10T06:58:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:22 smithi174 ceph-mon[39523]: osdmap e640: 8 total, 8 up, 8 in 2024-02-10T06:58:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: osdmap e641: 8 total, 8 up, 8 in 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-128"}]: dispatch 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: pgmap v982: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-128"}]': finished 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: osdmap e642: 8 total, 8 up, 8 in 2024-02-10T06:58:24.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-128", "mode": "writeback"}]: dispatch 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: osdmap e641: 8 total, 8 up, 8 in 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-128"}]: dispatch 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: pgmap v982: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-128"}]': finished 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: osdmap e642: 8 total, 8 up, 8 in 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-128", "mode": "writeback"}]: dispatch 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: osdmap e641: 8 total, 8 up, 8 in 2024-02-10T06:58:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-128"}]: dispatch 2024-02-10T06:58:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: pgmap v982: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-128"}]': finished 2024-02-10T06:58:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: osdmap e642: 8 total, 8 up, 8 in 2024-02-10T06:58:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-128", "mode": "writeback"}]: dispatch 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-128", "mode": "writeback"}]': finished 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[29878]: osdmap e643: 8 total, 8 up, 8 in 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[29878]: pgmap v985: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-128", "mode": "writeback"}]': finished 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[39523]: osdmap e643: 8 total, 8 up, 8 in 2024-02-10T06:58:26.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:25 smithi174 ceph-mon[39523]: pgmap v985: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:25 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-128", "mode": "writeback"}]': finished 2024-02-10T06:58:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:25 smithi184 ceph-mon[33513]: osdmap e643: 8 total, 8 up, 8 in 2024-02-10T06:58:26.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:25 smithi184 ceph-mon[33513]: pgmap v985: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:26.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:58:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:58:27.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:26 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:27.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:26 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:26 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:28.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:27 smithi174 ceph-mon[29878]: pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 397 B/s wr, 1 op/s 2024-02-10T06:58:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:58:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:58:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:27 smithi174 ceph-mon[39523]: pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 397 B/s wr, 1 op/s 2024-02-10T06:58:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:27 smithi184 ceph-mon[33513]: pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 397 B/s wr, 1 op/s 2024-02-10T06:58:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:30 smithi174 ceph-mon[29878]: pgmap v987: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 998 B/s rd, 332 B/s wr, 1 op/s 2024-02-10T06:58:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:30 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:30 smithi174 ceph-mon[39523]: pgmap v987: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 998 B/s rd, 332 B/s wr, 1 op/s 2024-02-10T06:58:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:30 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:30 smithi184 ceph-mon[33513]: pgmap v987: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 998 B/s rd, 332 B/s wr, 1 op/s 2024-02-10T06:58:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:30 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:30.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[29878]: osdmap e644: 8 total, 8 up, 8 in 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128"}]: dispatch 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[29878]: pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 286 B/s wr, 2 op/s 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[39523]: osdmap e644: 8 total, 8 up, 8 in 2024-02-10T06:58:32.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128"}]: dispatch 2024-02-10T06:58:32.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:31 smithi174 ceph-mon[39523]: pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 286 B/s wr, 2 op/s 2024-02-10T06:58:32.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:31 smithi184 ceph-mon[33513]: osdmap e644: 8 total, 8 up, 8 in 2024-02-10T06:58:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128"}]: dispatch 2024-02-10T06:58:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:31 smithi184 ceph-mon[33513]: pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 286 B/s wr, 2 op/s 2024-02-10T06:58:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:32 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128"}]': finished 2024-02-10T06:58:33.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:32 smithi174 ceph-mon[29878]: osdmap e645: 8 total, 8 up, 8 in 2024-02-10T06:58:33.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:32 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:33.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128"}]': finished 2024-02-10T06:58:33.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:32 smithi174 ceph-mon[39523]: osdmap e645: 8 total, 8 up, 8 in 2024-02-10T06:58:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:32 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-128"}]': finished 2024-02-10T06:58:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:32 smithi184 ceph-mon[33513]: osdmap e645: 8 total, 8 up, 8 in 2024-02-10T06:58:34.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:33 smithi174 ceph-mon[29878]: osdmap e646: 8 total, 8 up, 8 in 2024-02-10T06:58:34.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:33 smithi174 ceph-mon[29878]: pgmap v992: 172 pgs: 172 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-10T06:58:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:33 smithi174 ceph-mon[39523]: osdmap e646: 8 total, 8 up, 8 in 2024-02-10T06:58:34.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:33 smithi174 ceph-mon[39523]: pgmap v992: 172 pgs: 172 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-10T06:58:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:33 smithi184 ceph-mon[33513]: osdmap e646: 8 total, 8 up, 8 in 2024-02-10T06:58:34.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:33 smithi184 ceph-mon[33513]: pgmap v992: 172 pgs: 172 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-10T06:58:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:34 smithi184 ceph-mon[33513]: osdmap e647: 8 total, 8 up, 8 in 2024-02-10T06:58:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:34 smithi174 ceph-mon[29878]: osdmap e647: 8 total, 8 up, 8 in 2024-02-10T06:58:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:34 smithi174 ceph-mon[39523]: osdmap e647: 8 total, 8 up, 8 in 2024-02-10T06:58:35.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: osdmap e648: 8 total, 8 up, 8 in 2024-02-10T06:58:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:36.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: osdmap e649: 8 total, 8 up, 8 in 2024-02-10T06:58:36.275 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-130"}]: dispatch 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: osdmap e648: 8 total, 8 up, 8 in 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: osdmap e649: 8 total, 8 up, 8 in 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-130"}]: dispatch 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: osdmap e648: 8 total, 8 up, 8 in 2024-02-10T06:58:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: osdmap e649: 8 total, 8 up, 8 in 2024-02-10T06:58:36.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-130"}]: dispatch 2024-02-10T06:58:36.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:58:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:58:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-130"}]': finished 2024-02-10T06:58:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:37 smithi184 ceph-mon[33513]: osdmap e650: 8 total, 8 up, 8 in 2024-02-10T06:58:37.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-130", "mode": "writeback"}]: dispatch 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-130"}]': finished 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[29878]: osdmap e650: 8 total, 8 up, 8 in 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-130", "mode": "writeback"}]: dispatch 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-130"}]': finished 2024-02-10T06:58:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[39523]: osdmap e650: 8 total, 8 up, 8 in 2024-02-10T06:58:37.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-130", "mode": "writeback"}]: dispatch 2024-02-10T06:58:38.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[29878]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-130", "mode": "writeback"}]': finished 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[29878]: osdmap e651: 8 total, 8 up, 8 in 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:58:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:58:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[39523]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-130", "mode": "writeback"}]': finished 2024-02-10T06:58:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:38 smithi174 ceph-mon[39523]: osdmap e651: 8 total, 8 up, 8 in 2024-02-10T06:58:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:38 smithi184 ceph-mon[33513]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:38 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-130", "mode": "writeback"}]': finished 2024-02-10T06:58:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:38 smithi184 ceph-mon[33513]: osdmap e651: 8 total, 8 up, 8 in 2024-02-10T06:58:39.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:39.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:40.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:40 smithi184 ceph-mon[33513]: osdmap e652: 8 total, 8 up, 8 in 2024-02-10T06:58:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:40 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130"}]: dispatch 2024-02-10T06:58:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:40 smithi184 ceph-mon[33513]: pgmap v1001: 204 pgs: 204 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:40.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[29878]: osdmap e652: 8 total, 8 up, 8 in 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130"}]: dispatch 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[29878]: pgmap v1001: 204 pgs: 204 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[39523]: osdmap e652: 8 total, 8 up, 8 in 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130"}]: dispatch 2024-02-10T06:58:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:40 smithi174 ceph-mon[39523]: pgmap v1001: 204 pgs: 204 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:41.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:41 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:41.664 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130"}]': finished 2024-02-10T06:58:41.664 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:41 smithi184 ceph-mon[33513]: osdmap e653: 8 total, 8 up, 8 in 2024-02-10T06:58:41.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:41 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130"}]': finished 2024-02-10T06:58:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:41 smithi174 ceph-mon[29878]: osdmap e653: 8 total, 8 up, 8 in 2024-02-10T06:58:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:41 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-130"}]': finished 2024-02-10T06:58:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:41 smithi174 ceph-mon[39523]: osdmap e653: 8 total, 8 up, 8 in 2024-02-10T06:58:42.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:42 smithi184 ceph-mon[33513]: osdmap e654: 8 total, 8 up, 8 in 2024-02-10T06:58:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:42 smithi184 ceph-mon[33513]: pgmap v1004: 172 pgs: 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:42 smithi184 ceph-mon[33513]: osdmap e655: 8 total, 8 up, 8 in 2024-02-10T06:58:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:42 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:58:42.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[39523]: osdmap e654: 8 total, 8 up, 8 in 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[39523]: pgmap v1004: 172 pgs: 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[39523]: osdmap e655: 8 total, 8 up, 8 in 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[29878]: osdmap e654: 8 total, 8 up, 8 in 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[29878]: pgmap v1004: 172 pgs: 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[29878]: osdmap e655: 8 total, 8 up, 8 in 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:42 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:58:43.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:43 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:58:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:43 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:58:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:43 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:43 smithi184 ceph-mon[33513]: osdmap e656: 8 total, 8 up, 8 in 2024-02-10T06:58:43.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[29878]: osdmap e656: 8 total, 8 up, 8 in 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:58:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[39523]: osdmap e656: 8 total, 8 up, 8 in 2024-02-10T06:58:43.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:44 smithi184 ceph-mon[33513]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:44 smithi184 ceph-mon[33513]: osdmap e657: 8 total, 8 up, 8 in 2024-02-10T06:58:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:44.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[29878]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[29878]: osdmap e657: 8 total, 8 up, 8 in 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[39523]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[39523]: osdmap e657: 8 total, 8 up, 8 in 2024-02-10T06:58:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:46 smithi184 ceph-mon[33513]: osdmap e658: 8 total, 8 up, 8 in 2024-02-10T06:58:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-132", "mode": "writeback"}]: dispatch 2024-02-10T06:58:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:46 smithi184 ceph-mon[33513]: pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[29878]: osdmap e658: 8 total, 8 up, 8 in 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-132", "mode": "writeback"}]: dispatch 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[29878]: pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[39523]: osdmap e658: 8 total, 8 up, 8 in 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-132", "mode": "writeback"}]: dispatch 2024-02-10T06:58:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:46 smithi174 ceph-mon[39523]: pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:47.056 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:58:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:58:47.376 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:47 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:47.376 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-132", "mode": "writeback"}]': finished 2024-02-10T06:58:47.376 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:47 smithi184 ceph-mon[33513]: osdmap e659: 8 total, 8 up, 8 in 2024-02-10T06:58:47.376 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-132", "mode": "writeback"}]': finished 2024-02-10T06:58:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[29878]: osdmap e659: 8 total, 8 up, 8 in 2024-02-10T06:58:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-132", "mode": "writeback"}]': finished 2024-02-10T06:58:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[39523]: osdmap e659: 8 total, 8 up, 8 in 2024-02-10T06:58:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:48.062 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:58:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:58:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:48 smithi184 ceph-mon[33513]: osdmap e660: 8 total, 8 up, 8 in 2024-02-10T06:58:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:48 smithi184 ceph-mon[33513]: pgmap v1013: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:48.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:48 smithi174 ceph-mon[29878]: osdmap e660: 8 total, 8 up, 8 in 2024-02-10T06:58:48.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:48 smithi174 ceph-mon[29878]: pgmap v1013: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:48.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:48 smithi174 ceph-mon[39523]: osdmap e660: 8 total, 8 up, 8 in 2024-02-10T06:58:48.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:48 smithi174 ceph-mon[39523]: pgmap v1013: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:49.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:49 smithi184 ceph-mon[33513]: osdmap e661: 8 total, 8 up, 8 in 2024-02-10T06:58:49.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:49 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:49.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:49 smithi174 ceph-mon[29878]: osdmap e661: 8 total, 8 up, 8 in 2024-02-10T06:58:49.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:49 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:49.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:49 smithi174 ceph-mon[39523]: osdmap e661: 8 total, 8 up, 8 in 2024-02-10T06:58:49.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:49 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:50.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:50 smithi184 ceph-mon[33513]: osdmap e662: 8 total, 8 up, 8 in 2024-02-10T06:58:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:50 smithi184 ceph-mon[33513]: pgmap v1016: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[29878]: osdmap e662: 8 total, 8 up, 8 in 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[29878]: pgmap v1016: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[39523]: osdmap e662: 8 total, 8 up, 8 in 2024-02-10T06:58:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:50.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:50 smithi174 ceph-mon[39523]: pgmap v1016: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:58:51.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:51 smithi184 ceph-mon[33513]: osdmap e663: 8 total, 8 up, 8 in 2024-02-10T06:58:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:51 smithi174 ceph-mon[29878]: osdmap e663: 8 total, 8 up, 8 in 2024-02-10T06:58:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:51 smithi174 ceph-mon[39523]: osdmap e663: 8 total, 8 up, 8 in 2024-02-10T06:58:52.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:52 smithi184 ceph-mon[33513]: osdmap e664: 8 total, 8 up, 8 in 2024-02-10T06:58:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:52 smithi184 ceph-mon[33513]: pgmap v1019: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T06:58:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:52.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[29878]: osdmap e664: 8 total, 8 up, 8 in 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[29878]: pgmap v1019: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[39523]: osdmap e664: 8 total, 8 up, 8 in 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[39523]: pgmap v1019: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:58:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:58:53.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:53 smithi184 ceph-mon[33513]: osdmap e665: 8 total, 8 up, 8 in 2024-02-10T06:58:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:53 smithi174 ceph-mon[29878]: osdmap e665: 8 total, 8 up, 8 in 2024-02-10T06:58:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:58:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:53 smithi174 ceph-mon[39523]: osdmap e665: 8 total, 8 up, 8 in 2024-02-10T06:58:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132"}]: dispatch 2024-02-10T06:58:54.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:54 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:54 smithi184 ceph-mon[33513]: osdmap e666: 8 total, 8 up, 8 in 2024-02-10T06:58:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:54 smithi184 ceph-mon[33513]: pgmap v1022: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T06:58:54.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[29878]: osdmap e666: 8 total, 8 up, 8 in 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[29878]: pgmap v1022: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-132"}]': finished 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[39523]: osdmap e666: 8 total, 8 up, 8 in 2024-02-10T06:58:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:54 smithi174 ceph-mon[39523]: pgmap v1022: 204 pgs: 204 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T06:58:55.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:55 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:55.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:55 smithi184 ceph-mon[33513]: osdmap e667: 8 total, 8 up, 8 in 2024-02-10T06:58:55.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:55 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:55 smithi174 ceph-mon[29878]: osdmap e667: 8 total, 8 up, 8 in 2024-02-10T06:58:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:55 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:58:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:55 smithi174 ceph-mon[39523]: osdmap e667: 8 total, 8 up, 8 in 2024-02-10T06:58:56.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:56 smithi184 ceph-mon[33513]: osdmap e668: 8 total, 8 up, 8 in 2024-02-10T06:58:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:56 smithi184 ceph-mon[33513]: pgmap v1025: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:56 smithi174 ceph-mon[29878]: osdmap e668: 8 total, 8 up, 8 in 2024-02-10T06:58:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:56 smithi174 ceph-mon[29878]: pgmap v1025: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:56 smithi174 ceph-mon[39523]: osdmap e668: 8 total, 8 up, 8 in 2024-02-10T06:58:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:58:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:56 smithi174 ceph-mon[39523]: pgmap v1025: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:58:57.117 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:58:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:58:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:57 smithi184 ceph-mon[33513]: osdmap e669: 8 total, 8 up, 8 in 2024-02-10T06:58:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[29878]: osdmap e669: 8 total, 8 up, 8 in 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[39523]: osdmap e669: 8 total, 8 up, 8 in 2024-02-10T06:58:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:58:57.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:58:58.131 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:58:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:58:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[29878]: osdmap e670: 8 total, 8 up, 8 in 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-134"}]: dispatch 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[29878]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[39523]: osdmap e670: 8 total, 8 up, 8 in 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-134"}]: dispatch 2024-02-10T06:58:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:58 smithi174 ceph-mon[39523]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T06:58:58.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:58:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:58 smithi184 ceph-mon[33513]: osdmap e670: 8 total, 8 up, 8 in 2024-02-10T06:58:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-134"}]: dispatch 2024-02-10T06:58:58.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:58 smithi184 ceph-mon[33513]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-134"}]': finished 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[29878]: osdmap e671: 8 total, 8 up, 8 in 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-134", "mode": "writeback"}]: dispatch 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-134", "mode": "writeback"}]': finished 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-134"}]': finished 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[39523]: osdmap e671: 8 total, 8 up, 8 in 2024-02-10T06:58:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-134", "mode": "writeback"}]: dispatch 2024-02-10T06:58:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:59.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:58:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-134", "mode": "writeback"}]': finished 2024-02-10T06:58:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-134"}]': finished 2024-02-10T06:58:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:59 smithi184 ceph-mon[33513]: osdmap e671: 8 total, 8 up, 8 in 2024-02-10T06:58:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-134", "mode": "writeback"}]: dispatch 2024-02-10T06:58:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:59 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:58:59.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:58:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-134", "mode": "writeback"}]': finished 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: osdmap e672: 8 total, 8 up, 8 in 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: pgmap v1031: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: osdmap e673: 8 total, 8 up, 8 in 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134"}]: dispatch 2024-02-10T06:59:00.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: osdmap e672: 8 total, 8 up, 8 in 2024-02-10T06:59:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: pgmap v1031: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: osdmap e673: 8 total, 8 up, 8 in 2024-02-10T06:59:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134"}]: dispatch 2024-02-10T06:59:00.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: osdmap e672: 8 total, 8 up, 8 in 2024-02-10T06:59:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: pgmap v1031: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: osdmap e673: 8 total, 8 up, 8 in 2024-02-10T06:59:00.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134"}]: dispatch 2024-02-10T06:59:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:01 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:01 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134"}]': finished 2024-02-10T06:59:01.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:01 smithi174 ceph-mon[29878]: osdmap e674: 8 total, 8 up, 8 in 2024-02-10T06:59:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:01 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:01 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134"}]': finished 2024-02-10T06:59:01.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:01 smithi174 ceph-mon[39523]: osdmap e674: 8 total, 8 up, 8 in 2024-02-10T06:59:01.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:01 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:01 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-134"}]': finished 2024-02-10T06:59:01.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:01 smithi184 ceph-mon[33513]: osdmap e674: 8 total, 8 up, 8 in 2024-02-10T06:59:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:02 smithi174 ceph-mon[29878]: pgmap v1034: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:59:02.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:02 smithi174 ceph-mon[29878]: osdmap e675: 8 total, 8 up, 8 in 2024-02-10T06:59:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:02 smithi174 ceph-mon[39523]: pgmap v1034: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:59:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:02 smithi174 ceph-mon[39523]: osdmap e675: 8 total, 8 up, 8 in 2024-02-10T06:59:02.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:02 smithi184 ceph-mon[33513]: pgmap v1034: 204 pgs: 204 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T06:59:02.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:02 smithi184 ceph-mon[33513]: osdmap e675: 8 total, 8 up, 8 in 2024-02-10T06:59:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:03 smithi174 ceph-mon[29878]: osdmap e676: 8 total, 8 up, 8 in 2024-02-10T06:59:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:04.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:03 smithi174 ceph-mon[29878]: pgmap v1037: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:59:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:03 smithi174 ceph-mon[39523]: osdmap e676: 8 total, 8 up, 8 in 2024-02-10T06:59:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:04.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:03 smithi174 ceph-mon[39523]: pgmap v1037: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:59:04.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:03 smithi184 ceph-mon[33513]: osdmap e676: 8 total, 8 up, 8 in 2024-02-10T06:59:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:04.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:03 smithi184 ceph-mon[33513]: pgmap v1037: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T06:59:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[29878]: osdmap e677: 8 total, 8 up, 8 in 2024-02-10T06:59:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:05.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:05.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:05.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[39523]: osdmap e677: 8 total, 8 up, 8 in 2024-02-10T06:59:05.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:05.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:04 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:05.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:04 smithi184 ceph-mon[33513]: osdmap e677: 8 total, 8 up, 8 in 2024-02-10T06:59:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:05.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:04 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:05 smithi184 ceph-mon[33513]: osdmap e678: 8 total, 8 up, 8 in 2024-02-10T06:59:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-136"}]: dispatch 2024-02-10T06:59:06.274 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:05 smithi184 ceph-mon[33513]: pgmap v1040: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[29878]: osdmap e678: 8 total, 8 up, 8 in 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-136"}]: dispatch 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[29878]: pgmap v1040: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[39523]: osdmap e678: 8 total, 8 up, 8 in 2024-02-10T06:59:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-136"}]: dispatch 2024-02-10T06:59:06.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:05 smithi174 ceph-mon[39523]: pgmap v1040: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:59:06.993 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:59:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-136"}]': finished 2024-02-10T06:59:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-mon[33513]: osdmap e679: 8 total, 8 up, 8 in 2024-02-10T06:59:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-136", "mode": "writeback"}]: dispatch 2024-02-10T06:59:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:07.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:06 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:07.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-136"}]': finished 2024-02-10T06:59:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[29878]: osdmap e679: 8 total, 8 up, 8 in 2024-02-10T06:59:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-136", "mode": "writeback"}]: dispatch 2024-02-10T06:59:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-136"}]': finished 2024-02-10T06:59:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[39523]: osdmap e679: 8 total, 8 up, 8 in 2024-02-10T06:59:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-136", "mode": "writeback"}]: dispatch 2024-02-10T06:59:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:07.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:07.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:06 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:08.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:59:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-136", "mode": "writeback"}]': finished 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[29878]: osdmap e680: 8 total, 8 up, 8 in 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[29878]: pgmap v1043: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:59:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-136", "mode": "writeback"}]': finished 2024-02-10T06:59:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[39523]: osdmap e680: 8 total, 8 up, 8 in 2024-02-10T06:59:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:07 smithi174 ceph-mon[39523]: pgmap v1043: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:08.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:07 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:59:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-136", "mode": "writeback"}]': finished 2024-02-10T06:59:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:07 smithi184 ceph-mon[33513]: osdmap e680: 8 total, 8 up, 8 in 2024-02-10T06:59:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:07 smithi184 ceph-mon[33513]: pgmap v1043: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:09.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:09.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:09 smithi184 ceph-mon[33513]: osdmap e681: 8 total, 8 up, 8 in 2024-02-10T06:59:09.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136"}]: dispatch 2024-02-10T06:59:09.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:09 smithi174 ceph-mon[29878]: osdmap e681: 8 total, 8 up, 8 in 2024-02-10T06:59:09.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136"}]: dispatch 2024-02-10T06:59:09.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:09.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:09 smithi174 ceph-mon[39523]: osdmap e681: 8 total, 8 up, 8 in 2024-02-10T06:59:09.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136"}]: dispatch 2024-02-10T06:59:10.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:10 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136"}]': finished 2024-02-10T06:59:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:10 smithi184 ceph-mon[33513]: osdmap e682: 8 total, 8 up, 8 in 2024-02-10T06:59:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:10 smithi184 ceph-mon[33513]: pgmap v1046: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:10.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:10 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:10.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136"}]': finished 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[29878]: osdmap e682: 8 total, 8 up, 8 in 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[29878]: pgmap v1046: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-136"}]': finished 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[39523]: osdmap e682: 8 total, 8 up, 8 in 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[39523]: pgmap v1046: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:10.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:10 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:11.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:11 smithi184 ceph-mon[33513]: osdmap e683: 8 total, 8 up, 8 in 2024-02-10T06:59:11.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:11 smithi174 ceph-mon[29878]: osdmap e683: 8 total, 8 up, 8 in 2024-02-10T06:59:11.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:11 smithi174 ceph-mon[39523]: osdmap e683: 8 total, 8 up, 8 in 2024-02-10T06:59:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:12 smithi184 ceph-mon[33513]: osdmap e684: 8 total, 8 up, 8 in 2024-02-10T06:59:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:12.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:12 smithi184 ceph-mon[33513]: pgmap v1049: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T06:59:12.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:12 smithi174 ceph-mon[29878]: osdmap e684: 8 total, 8 up, 8 in 2024-02-10T06:59:12.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:12.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:12 smithi174 ceph-mon[29878]: pgmap v1049: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T06:59:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:12 smithi174 ceph-mon[39523]: osdmap e684: 8 total, 8 up, 8 in 2024-02-10T06:59:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:12.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:12 smithi174 ceph-mon[39523]: pgmap v1049: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T06:59:13.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:13 smithi184 ceph-mon[33513]: osdmap e685: 8 total, 8 up, 8 in 2024-02-10T06:59:13.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:13.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:13 smithi174 ceph-mon[29878]: osdmap e685: 8 total, 8 up, 8 in 2024-02-10T06:59:13.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:13 smithi174 ceph-mon[39523]: osdmap e685: 8 total, 8 up, 8 in 2024-02-10T06:59:13.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:14.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:14 smithi184 ceph-mon[33513]: osdmap e686: 8 total, 8 up, 8 in 2024-02-10T06:59:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:59:14.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:14 smithi184 ceph-mon[33513]: pgmap v1052: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T06:59:14.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[29878]: osdmap e686: 8 total, 8 up, 8 in 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[29878]: pgmap v1052: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[39523]: osdmap e686: 8 total, 8 up, 8 in 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:59:14.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:14 smithi174 ceph-mon[39523]: pgmap v1052: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T06:59:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:59:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:15 smithi184 ceph-mon[33513]: osdmap e687: 8 total, 8 up, 8 in 2024-02-10T06:59:15.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:59:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:59:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:15 smithi174 ceph-mon[29878]: osdmap e687: 8 total, 8 up, 8 in 2024-02-10T06:59:15.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:59:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:59:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:15 smithi174 ceph-mon[39523]: osdmap e687: 8 total, 8 up, 8 in 2024-02-10T06:59:15.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:59:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:59:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-mon[33513]: osdmap e688: 8 total, 8 up, 8 in 2024-02-10T06:59:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:59:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-mon[33513]: pgmap v1055: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:59:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:59:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-mon[33513]: osdmap e689: 8 total, 8 up, 8 in 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[29878]: osdmap e688: 8 total, 8 up, 8 in 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[29878]: pgmap v1055: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[29878]: osdmap e689: 8 total, 8 up, 8 in 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[39523]: osdmap e688: 8 total, 8 up, 8 in 2024-02-10T06:59:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T06:59:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[39523]: pgmap v1055: 204 pgs: 2 creating+peering, 30 unknown, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-02-10T06:59:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T06:59:16.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:16 smithi174 ceph-mon[39523]: osdmap e689: 8 total, 8 up, 8 in 2024-02-10T06:59:17.070 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:59:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:59:17.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:17.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:18.090 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:59:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:59:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:59:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138"}]: dispatch 2024-02-10T06:59:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:18 smithi184 ceph-mon[33513]: pgmap v1057: 204 pgs: 204 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138"}]: dispatch 2024-02-10T06:59:18.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:18 smithi174 ceph-mon[29878]: pgmap v1057: 204 pgs: 204 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138"}]: dispatch 2024-02-10T06:59:18.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:18 smithi174 ceph-mon[39523]: pgmap v1057: 204 pgs: 204 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:19.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138"}]': finished 2024-02-10T06:59:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:19 smithi184 ceph-mon[33513]: osdmap e690: 8 total, 8 up, 8 in 2024-02-10T06:59:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138"}]': finished 2024-02-10T06:59:19.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:19 smithi174 ceph-mon[29878]: osdmap e690: 8 total, 8 up, 8 in 2024-02-10T06:59:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-138"}]': finished 2024-02-10T06:59:19.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:19 smithi174 ceph-mon[39523]: osdmap e690: 8 total, 8 up, 8 in 2024-02-10T06:59:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:20 smithi184 ceph-mon[33513]: osdmap e691: 8 total, 8 up, 8 in 2024-02-10T06:59:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:20 smithi184 ceph-mon[33513]: pgmap v1060: 172 pgs: 172 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:20.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:20 smithi174 ceph-mon[29878]: osdmap e691: 8 total, 8 up, 8 in 2024-02-10T06:59:20.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:20 smithi174 ceph-mon[29878]: pgmap v1060: 172 pgs: 172 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:20.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:20 smithi174 ceph-mon[39523]: osdmap e691: 8 total, 8 up, 8 in 2024-02-10T06:59:20.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:20 smithi174 ceph-mon[39523]: pgmap v1060: 172 pgs: 172 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:21.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:21 smithi184 ceph-mon[33513]: osdmap e692: 8 total, 8 up, 8 in 2024-02-10T06:59:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:21 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:21.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:21 smithi174 ceph-mon[29878]: osdmap e692: 8 total, 8 up, 8 in 2024-02-10T06:59:21.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:21 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:21.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:21 smithi174 ceph-mon[39523]: osdmap e692: 8 total, 8 up, 8 in 2024-02-10T06:59:21.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:21 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:22 smithi184 ceph-mon[33513]: osdmap e693: 8 total, 8 up, 8 in 2024-02-10T06:59:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:22 smithi184 ceph-mon[33513]: pgmap v1063: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:22.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[39523]: osdmap e693: 8 total, 8 up, 8 in 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[39523]: pgmap v1063: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[29878]: osdmap e693: 8 total, 8 up, 8 in 2024-02-10T06:59:22.492 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:22.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[29878]: pgmap v1063: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:22.493 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:23 smithi174 ceph-mon[39523]: osdmap e694: 8 total, 8 up, 8 in 2024-02-10T06:59:23.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:23 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:59:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:23 smithi174 ceph-mon[29878]: osdmap e694: 8 total, 8 up, 8 in 2024-02-10T06:59:23.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:23 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:59:23.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:23 smithi184 ceph-mon[33513]: osdmap e694: 8 total, 8 up, 8 in 2024-02-10T06:59:23.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:23 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_count","val": "3"}]': finished 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[29878]: osdmap e695: 8 total, 8 up, 8 in 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[29878]: pgmap v1066: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_period","val": "3"}]': finished 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[29878]: osdmap e696: 8 total, 8 up, 8 in 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_count","val": "3"}]': finished 2024-02-10T06:59:24.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[39523]: osdmap e695: 8 total, 8 up, 8 in 2024-02-10T06:59:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:59:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[39523]: pgmap v1066: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_period","val": "3"}]': finished 2024-02-10T06:59:24.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:24 smithi174 ceph-mon[39523]: osdmap e696: 8 total, 8 up, 8 in 2024-02-10T06:59:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_count","val": "3"}]': finished 2024-02-10T06:59:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:24 smithi184 ceph-mon[33513]: osdmap e695: 8 total, 8 up, 8 in 2024-02-10T06:59:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T06:59:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:24 smithi184 ceph-mon[33513]: pgmap v1066: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:24 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_period","val": "3"}]': finished 2024-02-10T06:59:24.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:24 smithi184 ceph-mon[33513]: osdmap e696: 8 total, 8 up, 8 in 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[39523]: osdmap e697: 8 total, 8 up, 8 in 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:59:25.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[29878]: osdmap e697: 8 total, 8 up, 8 in 2024-02-10T06:59:25.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:25 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:59:25.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:59:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:59:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:25 smithi184 ceph-mon[33513]: osdmap e697: 8 total, 8 up, 8 in 2024-02-10T06:59:25.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:25 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T06:59:26.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:26 smithi174 ceph-mon[29878]: pgmap v1069: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:26 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T06:59:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:26 smithi174 ceph-mon[29878]: osdmap e698: 8 total, 8 up, 8 in 2024-02-10T06:59:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:26 smithi174 ceph-mon[39523]: pgmap v1069: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:26 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T06:59:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:26 smithi174 ceph-mon[39523]: osdmap e698: 8 total, 8 up, 8 in 2024-02-10T06:59:26.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:26 smithi184 ceph-mon[33513]: pgmap v1069: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:26 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T06:59:26.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:26 smithi184 ceph-mon[33513]: osdmap e698: 8 total, 8 up, 8 in 2024-02-10T06:59:27.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:59:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:59:27.488 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:27.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:28.141 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:59:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:59:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:59:28.488 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:28 smithi174 ceph-mon[39523]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:28.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:28 smithi174 ceph-mon[29878]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:28.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:28 smithi184 ceph-mon[33513]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:30.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:30 smithi174 ceph-mon[29878]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:59:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:30 smithi174 ceph-mon[39523]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:59:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:30 smithi184 ceph-mon[33513]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T06:59:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:32 smithi174 ceph-mon[29878]: pgmap v1073: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 5.8 KiB/s wr, 2 op/s 2024-02-10T06:59:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:32 smithi174 ceph-mon[39523]: pgmap v1073: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 5.8 KiB/s wr, 2 op/s 2024-02-10T06:59:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:32 smithi184 ceph-mon[33513]: pgmap v1073: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 5.8 KiB/s wr, 2 op/s 2024-02-10T06:59:34.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:34 smithi174 ceph-mon[29878]: pgmap v1074: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-02-10T06:59:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:34 smithi174 ceph-mon[39523]: pgmap v1074: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-02-10T06:59:34.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:34 smithi184 ceph-mon[33513]: pgmap v1074: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-02-10T06:59:36.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:35 smithi174 ceph-mon[29878]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-10T06:59:36.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:35 smithi174 ceph-mon[39523]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-10T06:59:36.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:35 smithi184 ceph-mon[33513]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-10T06:59:36.983 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:59:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:59:37.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:36 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:37.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:36 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:36 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:37.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:36 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:37.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:36 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:37.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:36 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:38.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:37 smithi174 ceph-mon[29878]: pgmap v1076: 204 pgs: 204 active+clean; 4.5 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 7.5 KiB/s wr, 2 op/s 2024-02-10T06:59:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:37 smithi174 ceph-mon[39523]: pgmap v1076: 204 pgs: 204 active+clean; 4.5 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 7.5 KiB/s wr, 2 op/s 2024-02-10T06:59:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:59:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:59:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:59:38.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:37 smithi184 ceph-mon[33513]: pgmap v1076: 204 pgs: 204 active+clean; 4.5 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 927 B/s rd, 7.5 KiB/s wr, 2 op/s 2024-02-10T06:59:39.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:39.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140"}]: dispatch 2024-02-10T06:59:39.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:39.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140"}]: dispatch 2024-02-10T06:59:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:39.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140"}]: dispatch 2024-02-10T06:59:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140"}]': finished 2024-02-10T06:59:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:39 smithi184 ceph-mon[33513]: osdmap e699: 8 total, 8 up, 8 in 2024-02-10T06:59:40.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:39 smithi184 ceph-mon[33513]: pgmap v1078: 204 pgs: 204 active+clean; 4.5 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-02-10T06:59:40.488 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140"}]': finished 2024-02-10T06:59:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:39 smithi174 ceph-mon[39523]: osdmap e699: 8 total, 8 up, 8 in 2024-02-10T06:59:40.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:39 smithi174 ceph-mon[39523]: pgmap v1078: 204 pgs: 204 active+clean; 4.5 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-02-10T06:59:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-140"}]': finished 2024-02-10T06:59:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:39 smithi174 ceph-mon[29878]: osdmap e699: 8 total, 8 up, 8 in 2024-02-10T06:59:40.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:39 smithi174 ceph-mon[29878]: pgmap v1078: 204 pgs: 204 active+clean; 4.5 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-02-10T06:59:41.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:41 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:41.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:41 smithi184 ceph-mon[33513]: osdmap e700: 8 total, 8 up, 8 in 2024-02-10T06:59:41.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:40 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:41.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:40 smithi174 ceph-mon[29878]: osdmap e700: 8 total, 8 up, 8 in 2024-02-10T06:59:41.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:41 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:41.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:41 smithi174 ceph-mon[39523]: osdmap e700: 8 total, 8 up, 8 in 2024-02-10T06:59:42.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:42 smithi184 ceph-mon[33513]: osdmap e701: 8 total, 8 up, 8 in 2024-02-10T06:59:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:42.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:42 smithi184 ceph-mon[33513]: pgmap v1081: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:59:42.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:42 smithi174 ceph-mon[29878]: osdmap e701: 8 total, 8 up, 8 in 2024-02-10T06:59:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:42.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:42 smithi174 ceph-mon[29878]: pgmap v1081: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:59:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:42 smithi174 ceph-mon[39523]: osdmap e701: 8 total, 8 up, 8 in 2024-02-10T06:59:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:42.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:42 smithi174 ceph-mon[39523]: pgmap v1081: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:59:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:43.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:43 smithi184 ceph-mon[33513]: osdmap e702: 8 total, 8 up, 8 in 2024-02-10T06:59:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:43 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:43.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:43 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:59:43.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:43.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[29878]: osdmap e702: 8 total, 8 up, 8 in 2024-02-10T06:59:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:43.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:59:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[39523]: osdmap e702: 8 total, 8 up, 8 in 2024-02-10T06:59:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:43.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:43 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T06:59:44.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: osdmap e703: 8 total, 8 up, 8 in 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: pgmap v1084: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-142"}]': finished 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: osdmap e704: 8 total, 8 up, 8 in 2024-02-10T06:59:44.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-142", "mode": "writeback"}]: dispatch 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: osdmap e703: 8 total, 8 up, 8 in 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: pgmap v1084: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-142"}]': finished 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: osdmap e704: 8 total, 8 up, 8 in 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-142", "mode": "writeback"}]: dispatch 2024-02-10T06:59:44.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: osdmap e703: 8 total, 8 up, 8 in 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: pgmap v1084: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-142"}]': finished 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: osdmap e704: 8 total, 8 up, 8 in 2024-02-10T06:59:44.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-142", "mode": "writeback"}]: dispatch 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-142", "mode": "writeback"}]': finished 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: osdmap e705: 8 total, 8 up, 8 in 2024-02-10T06:59:45.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:59:45.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-142", "mode": "writeback"}]': finished 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: osdmap e705: 8 total, 8 up, 8 in 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T06:59:45.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-142", "mode": "writeback"}]': finished 2024-02-10T06:59:45.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: osdmap e705: 8 total, 8 up, 8 in 2024-02-10T06:59:45.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T06:59:46.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:46 smithi184 ceph-mon[33513]: pgmap v1087: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:59:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:46 smithi184 ceph-mon[33513]: osdmap e706: 8 total, 8 up, 8 in 2024-02-10T06:59:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:59:46.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[29878]: pgmap v1087: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[29878]: osdmap e706: 8 total, 8 up, 8 in 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[39523]: pgmap v1087: 204 pgs: 24 creating+peering, 8 unknown, 172 active+clean; 4.4 MiB data, 741 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_count","val": "2"}]': finished 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[39523]: osdmap e706: 8 total, 8 up, 8 in 2024-02-10T06:59:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T06:59:47.034 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:59:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:59:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:59:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:47 smithi184 ceph-mon[33513]: osdmap e707: 8 total, 8 up, 8 in 2024-02-10T06:59:47.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[29878]: osdmap e707: 8 total, 8 up, 8 in 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_period","val": "600"}]': finished 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[39523]: osdmap e707: 8 total, 8 up, 8 in 2024-02-10T06:59:47.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T06:59:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:59:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:59:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:59:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[29878]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:59:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:59:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[29878]: osdmap e708: 8 total, 8 up, 8 in 2024-02-10T06:59:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:59:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[39523]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:59:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:59:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[39523]: osdmap e708: 8 total, 8 up, 8 in 2024-02-10T06:59:48.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:59:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:48 smithi184 ceph-mon[33513]: pgmap v1090: 204 pgs: 204 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:59:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:48 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T06:59:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T06:59:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:48 smithi184 ceph-mon[33513]: osdmap e708: 8 total, 8 up, 8 in 2024-02-10T06:59:48.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T06:59:50.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:59:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:50 smithi184 ceph-mon[33513]: osdmap e709: 8 total, 8 up, 8 in 2024-02-10T06:59:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:59:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:50 smithi184 ceph-mon[33513]: pgmap v1093: 204 pgs: 204 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:59:50.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:50 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:50.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[29878]: osdmap e709: 8 total, 8 up, 8 in 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[29878]: pgmap v1093: 204 pgs: 204 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[39523]: osdmap e709: 8 total, 8 up, 8 in 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[39523]: pgmap v1093: 204 pgs: 204 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T06:59:50.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:50 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T06:59:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:51 smithi184 ceph-mon[33513]: osdmap e710: 8 total, 8 up, 8 in 2024-02-10T06:59:51.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:59:51.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T06:59:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:51 smithi174 ceph-mon[29878]: osdmap e710: 8 total, 8 up, 8 in 2024-02-10T06:59:51.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:59:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T06:59:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:51 smithi174 ceph-mon[39523]: osdmap e710: 8 total, 8 up, 8 in 2024-02-10T06:59:51.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T06:59:52.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T06:59:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:52 smithi184 ceph-mon[33513]: osdmap e711: 8 total, 8 up, 8 in 2024-02-10T06:59:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:52 smithi184 ceph-mon[33513]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:52.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:52.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[29878]: osdmap e711: 8 total, 8 up, 8 in 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[29878]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[39523]: osdmap e711: 8 total, 8 up, 8 in 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[39523]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:52.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T06:59:53.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:53 smithi184 ceph-mon[33513]: osdmap e712: 8 total, 8 up, 8 in 2024-02-10T06:59:53.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:53 smithi174 ceph-mon[29878]: osdmap e712: 8 total, 8 up, 8 in 2024-02-10T06:59:53.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T06:59:53.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:53 smithi174 ceph-mon[39523]: osdmap e712: 8 total, 8 up, 8 in 2024-02-10T06:59:53.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:54.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]': finished 2024-02-10T06:59:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:54 smithi184 ceph-mon[33513]: osdmap e713: 8 total, 8 up, 8 in 2024-02-10T06:59:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:54.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:54 smithi184 ceph-mon[33513]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:54.488 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]': finished 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[39523]: osdmap e713: 8 total, 8 up, 8 in 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[39523]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]': finished 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[29878]: osdmap e713: 8 total, 8 up, 8 in 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-142"}]: dispatch 2024-02-10T06:59:54.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:54 smithi174 ceph-mon[29878]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T06:59:55.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:55 smithi184 ceph-mon[33513]: osdmap e714: 8 total, 8 up, 8 in 2024-02-10T06:59:55.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:55 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:55.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:55 smithi174 ceph-mon[29878]: osdmap e714: 8 total, 8 up, 8 in 2024-02-10T06:59:55.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:55 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:55 smithi174 ceph-mon[39523]: osdmap e714: 8 total, 8 up, 8 in 2024-02-10T06:59:55.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:55 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T06:59:56.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:56 smithi184 ceph-mon[33513]: osdmap e715: 8 total, 8 up, 8 in 2024-02-10T06:59:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:56 smithi184 ceph-mon[33513]: pgmap v1102: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:56 smithi174 ceph-mon[29878]: osdmap e715: 8 total, 8 up, 8 in 2024-02-10T06:59:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:56 smithi174 ceph-mon[29878]: pgmap v1102: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:56 smithi174 ceph-mon[39523]: osdmap e715: 8 total, 8 up, 8 in 2024-02-10T06:59:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T06:59:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:56 smithi174 ceph-mon[39523]: pgmap v1102: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-02-10T06:59:57.064 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 06:59:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T06:59:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:57 smithi184 ceph-mon[33513]: osdmap e716: 8 total, 8 up, 8 in 2024-02-10T06:59:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:57 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:57.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[29878]: osdmap e716: 8 total, 8 up, 8 in 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[39523]: osdmap e716: 8 total, 8 up, 8 in 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T06:59:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T06:59:58.077 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 06:59:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:06:59:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T06:59:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:58 smithi184 ceph-mon[33513]: osdmap e717: 8 total, 8 up, 8 in 2024-02-10T06:59:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:58 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T06:59:58.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:58 smithi184 ceph-mon[33513]: pgmap v1105: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:59:58.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[29878]: osdmap e717: 8 total, 8 up, 8 in 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[29878]: pgmap v1105: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[39523]: osdmap e717: 8 total, 8 up, 8 in 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T06:59:58.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:58 smithi174 ceph-mon[39523]: pgmap v1105: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T06:59:59.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-144"}]': finished 2024-02-10T06:59:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:59 smithi184 ceph-mon[33513]: osdmap e718: 8 total, 8 up, 8 in 2024-02-10T06:59:59.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 06:59:59 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-144", "mode": "readproxy"}]: dispatch 2024-02-10T06:59:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-144"}]': finished 2024-02-10T06:59:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:59 smithi174 ceph-mon[29878]: osdmap e718: 8 total, 8 up, 8 in 2024-02-10T06:59:59.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 06:59:59 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-144", "mode": "readproxy"}]: dispatch 2024-02-10T06:59:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-144"}]': finished 2024-02-10T06:59:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:59 smithi174 ceph-mon[39523]: osdmap e718: 8 total, 8 up, 8 in 2024-02-10T06:59:59.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 06:59:59 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-144", "mode": "readproxy"}]: dispatch 2024-02-10T07:00:00.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-144", "mode": "readproxy"}]': finished 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: osdmap e719: 8 total, 8 up, 8 in 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: pgmap v1108: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 2 pool(s) do not have an application enabled 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: pool 'test-rados-api-smithi174-79953-144' with cache_mode readproxy needs hit_set_type to be set but it is not 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: application not enabled on pool 'test-rados-api-smithi174-79953-111' 2024-02-10T07:00:00.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:00 smithi184 ceph-mon[33513]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-144", "mode": "readproxy"}]': finished 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: osdmap e719: 8 total, 8 up, 8 in 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: pgmap v1108: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 2 pool(s) do not have an application enabled 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: pool 'test-rados-api-smithi174-79953-144' with cache_mode readproxy needs hit_set_type to be set but it is not 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T07:00:00.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: application not enabled on pool 'test-rados-api-smithi174-79953-111' 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[29878]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-144", "mode": "readproxy"}]': finished 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: osdmap e719: 8 total, 8 up, 8 in 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: pgmap v1108: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 2 pool(s) do not have an application enabled 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: pool 'test-rados-api-smithi174-79953-144' with cache_mode readproxy needs hit_set_type to be set but it is not 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T07:00:00.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: application not enabled on pool 'test-rados-api-smithi174-79953-111' 2024-02-10T07:00:00.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:00 smithi174 ceph-mon[39523]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T07:00:02.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:02 smithi174 ceph-mon[29878]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 200 B/s wr, 1 op/s 2024-02-10T07:00:02.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:02 smithi174 ceph-mon[39523]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 200 B/s wr, 1 op/s 2024-02-10T07:00:02.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:02 smithi184 ceph-mon[33513]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 200 B/s wr, 1 op/s 2024-02-10T07:00:04.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:04 smithi174 ceph-mon[29878]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1008 B/s rd, 168 B/s wr, 1 op/s 2024-02-10T07:00:04.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:04 smithi174 ceph-mon[39523]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1008 B/s rd, 168 B/s wr, 1 op/s 2024-02-10T07:00:04.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:04 smithi184 ceph-mon[33513]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1008 B/s rd, 168 B/s wr, 1 op/s 2024-02-10T07:00:06.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:05 smithi174 ceph-mon[29878]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 127 B/s wr, 0 op/s 2024-02-10T07:00:06.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:05 smithi174 ceph-mon[39523]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 127 B/s wr, 0 op/s 2024-02-10T07:00:06.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:05 smithi184 ceph-mon[33513]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 127 B/s wr, 0 op/s 2024-02-10T07:00:07.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:00:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:00:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:08.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:08.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:00:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:00:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:08.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:09.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:08 smithi184 ceph-mon[33513]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 B/s wr, 1 op/s 2024-02-10T07:00:09.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:08 smithi174 ceph-mon[29878]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 B/s wr, 1 op/s 2024-02-10T07:00:09.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:08 smithi174 ceph-mon[39523]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 B/s wr, 1 op/s 2024-02-10T07:00:10.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:10.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:10.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:11.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:10 smithi184 ceph-mon[33513]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 101 B/s wr, 1 op/s 2024-02-10T07:00:11.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:11.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:10 smithi184 ceph-mon[33513]: osdmap e720: 8 total, 8 up, 8 in 2024-02-10T07:00:11.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[29878]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 101 B/s wr, 1 op/s 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[29878]: osdmap e720: 8 total, 8 up, 8 in 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[39523]: pgmap v1113: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 101 B/s wr, 1 op/s 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[39523]: osdmap e720: 8 total, 8 up, 8 in 2024-02-10T07:00:11.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T07:00:12.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:11 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:00:12.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]': finished 2024-02-10T07:00:12.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:11 smithi184 ceph-mon[33513]: osdmap e721: 8 total, 8 up, 8 in 2024-02-10T07:00:12.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:12.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]': finished 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[29878]: osdmap e721: 8 total, 8 up, 8 in 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]': finished 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[39523]: osdmap e721: 8 total, 8 up, 8 in 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:12.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-144"}]: dispatch 2024-02-10T07:00:13.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:12 smithi184 ceph-mon[33513]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:13.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:12 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:13.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:12 smithi184 ceph-mon[33513]: osdmap e722: 8 total, 8 up, 8 in 2024-02-10T07:00:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:12 smithi174 ceph-mon[29878]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:12 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:13.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:12 smithi174 ceph-mon[29878]: osdmap e722: 8 total, 8 up, 8 in 2024-02-10T07:00:13.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:12 smithi174 ceph-mon[39523]: pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:13.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:12 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:13.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:12 smithi174 ceph-mon[39523]: osdmap e722: 8 total, 8 up, 8 in 2024-02-10T07:00:14.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:13 smithi184 ceph-mon[33513]: osdmap e723: 8 total, 8 up, 8 in 2024-02-10T07:00:14.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:14.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:13 smithi174 ceph-mon[29878]: osdmap e723: 8 total, 8 up, 8 in 2024-02-10T07:00:14.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:13 smithi174 ceph-mon[39523]: osdmap e723: 8 total, 8 up, 8 in 2024-02-10T07:00:14.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:15.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:14 smithi184 ceph-mon[33513]: pgmap v1119: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:15.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:15.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:14 smithi184 ceph-mon[33513]: osdmap e724: 8 total, 8 up, 8 in 2024-02-10T07:00:15.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[29878]: pgmap v1119: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[29878]: osdmap e724: 8 total, 8 up, 8 in 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[39523]: pgmap v1119: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[39523]: osdmap e724: 8 total, 8 up, 8 in 2024-02-10T07:00:15.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:00:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:00:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:15 smithi174 ceph-mon[29878]: osdmap e725: 8 total, 8 up, 8 in 2024-02-10T07:00:16.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:16.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:00:16.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:15 smithi174 ceph-mon[39523]: osdmap e725: 8 total, 8 up, 8 in 2024-02-10T07:00:16.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:00:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:15 smithi184 ceph-mon[33513]: osdmap e725: 8 total, 8 up, 8 in 2024-02-10T07:00:16.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:17.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: pgmap v1122: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-146"}]': finished 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: osdmap e726: 8 total, 8 up, 8 in 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-146", "mode": "writeback"}]: dispatch 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-146", "mode": "writeback"}]': finished 2024-02-10T07:00:17.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:16 smithi184 ceph-mon[33513]: osdmap e727: 8 total, 8 up, 8 in 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: pgmap v1122: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-146"}]': finished 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: osdmap e726: 8 total, 8 up, 8 in 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-146", "mode": "writeback"}]: dispatch 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-146", "mode": "writeback"}]': finished 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[29878]: osdmap e727: 8 total, 8 up, 8 in 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: pgmap v1122: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:17.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi174-79953-111", "overlaypool": "test-rados-api-smithi174-79953-146"}]': finished 2024-02-10T07:00:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: osdmap e726: 8 total, 8 up, 8 in 2024-02-10T07:00:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-146", "mode": "writeback"}]: dispatch 2024-02-10T07:00:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:00:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi174-79953-146", "mode": "writeback"}]': finished 2024-02-10T07:00:17.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:16 smithi174 ceph-mon[39523]: osdmap e727: 8 total, 8 up, 8 in 2024-02-10T07:00:18.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:18.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:17 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:00:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:00:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:00:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:00:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:17 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:00:18.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:18.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:17 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[39523]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[39523]: osdmap e728: 8 total, 8 up, 8 in 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[29878]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[29878]: osdmap e728: 8 total, 8 up, 8 in 2024-02-10T07:00:19.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:18 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:00:19.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:18 smithi184 ceph-mon[33513]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:00:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:00:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:18 smithi184 ceph-mon[33513]: osdmap e728: 8 total, 8 up, 8 in 2024-02-10T07:00:19.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:18 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: osdmap e729: 8 total, 8 up, 8 in 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: osdmap e730: 8 total, 8 up, 8 in 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: osdmap e729: 8 total, 8 up, 8 in 2024-02-10T07:00:20.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:00:20.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:00:20.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:00:20.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: osdmap e730: 8 total, 8 up, 8 in 2024-02-10T07:00:20.240 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:19 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:00:20.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:00:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: osdmap e729: 8 total, 8 up, 8 in 2024-02-10T07:00:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:00:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:00:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:00:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: osdmap e730: 8 total, 8 up, 8 in 2024-02-10T07:00:20.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:19 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[29878]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[29878]: osdmap e731: 8 total, 8 up, 8 in 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[39523]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:00:21.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:00:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[39523]: osdmap e731: 8 total, 8 up, 8 in 2024-02-10T07:00:21.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:20 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:00:21.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:20 smithi184 ceph-mon[33513]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:00:21.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:20 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:21.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:00:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:20 smithi184 ceph-mon[33513]: osdmap e731: 8 total, 8 up, 8 in 2024-02-10T07:00:21.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:20 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:00:23.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:22 smithi184 ceph-mon[33513]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:22 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:00:23.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:22 smithi184 ceph-mon[33513]: osdmap e732: 8 total, 8 up, 8 in 2024-02-10T07:00:23.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[29878]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[29878]: osdmap e732: 8 total, 8 up, 8 in 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[39523]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-146","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:00:23.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:22 smithi174 ceph-mon[39523]: osdmap e732: 8 total, 8 up, 8 in 2024-02-10T07:00:24.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:23 smithi174 ceph-mon[29878]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:00:24.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:23 smithi174 ceph-mon[39523]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:00:24.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:23 smithi184 ceph-mon[33513]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:00:25.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:24 smithi174 ceph-mon[29878]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:25.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:24 smithi174 ceph-mon[39523]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:25.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:24 smithi184 ceph-mon[33513]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:25 smithi184 ceph-mon[33513]: pgmap v1134: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:00:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:25 smithi174 ceph-mon[29878]: pgmap v1134: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:00:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:25 smithi174 ceph-mon[39523]: pgmap v1134: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:00:26.993 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:00:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:00:27.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:26 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:27.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:26 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:26 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:28 smithi174 ceph-mon[29878]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:28 smithi174 ceph-mon[39523]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:00:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:00:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:00:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:28 smithi184 ceph-mon[33513]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:00:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:30 smithi174 ceph-mon[29878]: pgmap v1136: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:00:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:30 smithi174 ceph-mon[39523]: pgmap v1136: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:00:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:30 smithi184 ceph-mon[33513]: pgmap v1136: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:00:32.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:32 smithi174 ceph-mon[29878]: pgmap v1137: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:00:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:32 smithi174 ceph-mon[39523]: pgmap v1137: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:00:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:32 smithi184 ceph-mon[33513]: pgmap v1137: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:00:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:33.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:33.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:33 smithi184 ceph-mon[33513]: osdmap e733: 8 total, 8 up, 8 in 2024-02-10T07:00:33.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:33.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:33 smithi174 ceph-mon[29878]: osdmap e733: 8 total, 8 up, 8 in 2024-02-10T07:00:33.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:33 smithi174 ceph-mon[39523]: osdmap e733: 8 total, 8 up, 8 in 2024-02-10T07:00:33.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:34 smithi184 ceph-mon[33513]: pgmap v1139: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:00:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]': finished 2024-02-10T07:00:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:34 smithi184 ceph-mon[33513]: osdmap e734: 8 total, 8 up, 8 in 2024-02-10T07:00:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:34.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:34 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:34.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[29878]: pgmap v1139: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]': finished 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[29878]: osdmap e734: 8 total, 8 up, 8 in 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[39523]: pgmap v1139: 204 pgs: 204 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]': finished 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[39523]: osdmap e734: 8 total, 8 up, 8 in 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:34 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-146"}]: dispatch 2024-02-10T07:00:35.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:35 smithi184 ceph-mon[33513]: osdmap e735: 8 total, 8 up, 8 in 2024-02-10T07:00:35.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:35 smithi174 ceph-mon[29878]: osdmap e735: 8 total, 8 up, 8 in 2024-02-10T07:00:35.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:35 smithi174 ceph-mon[39523]: osdmap e735: 8 total, 8 up, 8 in 2024-02-10T07:00:36.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:36 smithi184 ceph-mon[33513]: pgmap v1142: 172 pgs: 172 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T07:00:36.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:36 smithi184 ceph-mon[33513]: osdmap e736: 8 total, 8 up, 8 in 2024-02-10T07:00:36.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:36 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:36.524 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:36 smithi184 ceph-mon[33513]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[29878]: pgmap v1142: 172 pgs: 172 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[29878]: osdmap e736: 8 total, 8 up, 8 in 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[29878]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[39523]: pgmap v1142: 172 pgs: 172 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[39523]: osdmap e736: 8 total, 8 up, 8 in 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:36.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:36 smithi174 ceph-mon[39523]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:00:36.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:00:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:00:37.536 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:37.536 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:37 smithi184 ceph-mon[33513]: osdmap e737: 8 total, 8 up, 8 in 2024-02-10T07:00:37.536 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:37 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:00:37.536 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:37.536 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[29878]: osdmap e737: 8 total, 8 up, 8 in 2024-02-10T07:00:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:00:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:37.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:37.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:37.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[39523]: osdmap e737: 8 total, 8 up, 8 in 2024-02-10T07:00:37.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:00:37.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:37.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:38.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:00:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:00:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:00:38.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:38 smithi184 ceph-mon[33513]: pgmap v1145: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:00:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:00:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:38 smithi184 ceph-mon[33513]: osdmap e738: 8 total, 8 up, 8 in 2024-02-10T07:00:38.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:38 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]: dispatch 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[29878]: pgmap v1145: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[29878]: osdmap e738: 8 total, 8 up, 8 in 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]: dispatch 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[39523]: pgmap v1145: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[39523]: osdmap e738: 8 total, 8 up, 8 in 2024-02-10T07:00:38.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:38 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]: dispatch 2024-02-10T07:00:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]': finished 2024-02-10T07:00:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:39 smithi184 ceph-mon[33513]: osdmap e739: 8 total, 8 up, 8 in 2024-02-10T07:00:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:39.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:39 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]: dispatch 2024-02-10T07:00:39.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]': finished 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[29878]: osdmap e739: 8 total, 8 up, 8 in 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]: dispatch 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]': finished 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[39523]: osdmap e739: 8 total, 8 up, 8 in 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:39.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:39 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-148"}]: dispatch 2024-02-10T07:00:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:40 smithi184 ceph-mon[33513]: pgmap v1148: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:40 smithi184 ceph-mon[33513]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:40.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:40 smithi184 ceph-mon[33513]: osdmap e740: 8 total, 8 up, 8 in 2024-02-10T07:00:40.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:40 smithi174 ceph-mon[29878]: pgmap v1148: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:40 smithi174 ceph-mon[29878]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:40 smithi174 ceph-mon[29878]: osdmap e740: 8 total, 8 up, 8 in 2024-02-10T07:00:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:40 smithi174 ceph-mon[39523]: pgmap v1148: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:40 smithi174 ceph-mon[39523]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:40 smithi174 ceph-mon[39523]: osdmap e740: 8 total, 8 up, 8 in 2024-02-10T07:00:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:41 smithi184 ceph-mon[33513]: osdmap e741: 8 total, 8 up, 8 in 2024-02-10T07:00:41.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:41 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:41 smithi174 ceph-mon[29878]: osdmap e741: 8 total, 8 up, 8 in 2024-02-10T07:00:41.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:41 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:41 smithi174 ceph-mon[39523]: osdmap e741: 8 total, 8 up, 8 in 2024-02-10T07:00:41.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:41 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:42.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:42 smithi184 ceph-mon[33513]: pgmap v1151: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:00:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:42 smithi184 ceph-mon[33513]: osdmap e742: 8 total, 8 up, 8 in 2024-02-10T07:00:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:42.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:42 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-150"}]: dispatch 2024-02-10T07:00:42.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[29878]: pgmap v1151: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[29878]: osdmap e742: 8 total, 8 up, 8 in 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-150"}]: dispatch 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[39523]: pgmap v1151: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[39523]: osdmap e742: 8 total, 8 up, 8 in 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:42 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-150"}]: dispatch 2024-02-10T07:00:43.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:43 smithi184 ceph-mon[33513]: osdmap e743: 8 total, 8 up, 8 in 2024-02-10T07:00:43.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:43 smithi174 ceph-mon[29878]: osdmap e743: 8 total, 8 up, 8 in 2024-02-10T07:00:43.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:43 smithi174 ceph-mon[39523]: osdmap e743: 8 total, 8 up, 8 in 2024-02-10T07:00:44.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:44 smithi184 ceph-mon[33513]: pgmap v1154: 172 pgs: 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:00:44.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:44 smithi184 ceph-mon[33513]: osdmap e744: 8 total, 8 up, 8 in 2024-02-10T07:00:44.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:44 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:44.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:44 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:00:44.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[29878]: pgmap v1154: 172 pgs: 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[29878]: osdmap e744: 8 total, 8 up, 8 in 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[39523]: pgmap v1154: 172 pgs: 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[39523]: osdmap e744: 8 total, 8 up, 8 in 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:44 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:00:45.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:45.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: osdmap e745: 8 total, 8 up, 8 in 2024-02-10T07:00:45.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:45.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-152"}]: dispatch 2024-02-10T07:00:45.620 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:45.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:00:45.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:00:45.621 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: osdmap e745: 8 total, 8 up, 8 in 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-152"}]: dispatch 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:45.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: osdmap e745: 8 total, 8 up, 8 in 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-152"}]: dispatch 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:45.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:00:45.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:00:45.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:45 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:00:45.873 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:00:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a[66294]: logger=sqlstore.transactions t=2024-02-10T07:00:45.620258396Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-10T07:00:46.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:46 smithi184 ceph-mon[33513]: pgmap v1157: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:46.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:46 smithi184 ceph-mon[33513]: osdmap e746: 8 total, 8 up, 8 in 2024-02-10T07:00:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:46 smithi174 ceph-mon[29878]: pgmap v1157: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:46.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:46 smithi174 ceph-mon[29878]: osdmap e746: 8 total, 8 up, 8 in 2024-02-10T07:00:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:46 smithi174 ceph-mon[39523]: pgmap v1157: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:46.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:46 smithi174 ceph-mon[39523]: osdmap e746: 8 total, 8 up, 8 in 2024-02-10T07:00:47.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:00:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:00:47.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:47 smithi184 ceph-mon[33513]: osdmap e747: 8 total, 8 up, 8 in 2024-02-10T07:00:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:47.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:47.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-mon[29878]: osdmap e747: 8 total, 8 up, 8 in 2024-02-10T07:00:47.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:47.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-mon[39523]: osdmap e747: 8 total, 8 up, 8 in 2024-02-10T07:00:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:00:47.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:48.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:00:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:00:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:00:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:48 smithi184 ceph-mon[33513]: pgmap v1160: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 821 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:00:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:48 smithi184 ceph-mon[33513]: osdmap e748: 8 total, 8 up, 8 in 2024-02-10T07:00:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-111","var": "dedup_tier","val": "test-rados-api-smithi174-79953-154"}]: dispatch 2024-02-10T07:00:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:48.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-154"}]: dispatch 2024-02-10T07:00:48.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[29878]: pgmap v1160: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 821 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[29878]: osdmap e748: 8 total, 8 up, 8 in 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-111","var": "dedup_tier","val": "test-rados-api-smithi174-79953-154"}]: dispatch 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-154"}]: dispatch 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[39523]: pgmap v1160: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 821 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi174-79953-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[39523]: osdmap e748: 8 total, 8 up, 8 in 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi174-79953-111","var": "dedup_tier","val": "test-rados-api-smithi174-79953-154"}]: dispatch 2024-02-10T07:00:48.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:48.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi174-79953-111", "tierpool": "test-rados-api-smithi174-79953-154"}]: dispatch 2024-02-10T07:00:49.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:49 smithi184 ceph-mon[33513]: osdmap e749: 8 total, 8 up, 8 in 2024-02-10T07:00:49.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:49 smithi174 ceph-mon[29878]: osdmap e749: 8 total, 8 up, 8 in 2024-02-10T07:00:49.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:49 smithi174 ceph-mon[39523]: osdmap e749: 8 total, 8 up, 8 in 2024-02-10T07:00:50.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:50 smithi184 ceph-mon[33513]: pgmap v1163: 172 pgs: 172 active+clean; 4.4 MiB data, 821 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:00:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:50 smithi184 ceph-mon[33513]: osdmap e750: 8 total, 8 up, 8 in 2024-02-10T07:00:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:50 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:50.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:50 smithi174 ceph-mon[29878]: pgmap v1163: 172 pgs: 172 active+clean; 4.4 MiB data, 821 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:00:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:50 smithi174 ceph-mon[29878]: osdmap e750: 8 total, 8 up, 8 in 2024-02-10T07:00:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:50 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:50 smithi174 ceph-mon[39523]: pgmap v1163: 172 pgs: 172 active+clean; 4.4 MiB data, 821 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:00:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:50 smithi174 ceph-mon[39523]: osdmap e750: 8 total, 8 up, 8 in 2024-02-10T07:00:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:50 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7790 ms) 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13000 ms) 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8057 ms) 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13079 ms) 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7860 ms) 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-10T07:00:51.398 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8061 ms) 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: ok, hit_set contains 337:602f83fe:::foo:head 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9062 ms) 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first is 1707548367 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,1707548372,1707548373,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,1707548372,1707548373,0 2024-02-10T07:00:51.399 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,1707548372,1707548373,0 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,1707548372,1707548373,1707548375,1707548376,0 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,1707548372,1707548373,1707548375,1707548376,0 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548367,1707548369,1707548370,1707548372,1707548373,1707548375,1707548376,0 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: got ls 1707548370,1707548372,1707548373,1707548375,1707548376,1707548378,1707548379,0 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: first now 1707548370, trimmed 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20910 ms) 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: foo0 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14052 ms) 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17796 ms) 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-10T07:00:51.400 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22398 ms) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5045 ms) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3019 ms) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3027 ms) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3027 ms) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232040 ms total) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (864489 ms total) 2024-02-10T07:00:51.401 INFO:tasks.workunit.client.0.smithi174.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79664 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79664 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79880 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79880 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.406 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80379 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80379 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80099 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80099 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80007 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80007 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79740 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79740 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80065 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80065 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.407 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80442 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80442 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80513 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80513 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79768 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79768 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80397 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80397 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80332 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80332 2024-02-10T07:00:51.408 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79638 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79638 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79702 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79702 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80349 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80349 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79629 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79629 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79649 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79649 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.409 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80666 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80666 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79920 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79920 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80741 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80741 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80043 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80043 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80427 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80427 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.410 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80781 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80781 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79982 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79982 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80362 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80362 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80608 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80608 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80407 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80407 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80719 2024-02-10T07:00:51.411 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80719 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80274 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80274 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79819 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79819 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80762 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80762 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80152 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80152 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79622 2024-02-10T07:00:51.412 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79622 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79682 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79682 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80083 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80083 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=79724 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 79724 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80200 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80200 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80224 2024-02-10T07:00:51.413 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80224 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80568 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80568 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=80845 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 80845 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ exit 0 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ cleanup 2024-02-10T07:00:51.414 INFO:tasks.workunit.client.0.smithi174.stderr:+ pkill -P 79617 2024-02-10T07:00:51.415 INFO:tasks.workunit.client.0.smithi174.stderr:+ true 2024-02-10T07:00:51.416 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-10T07:00:51.416 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-10T07:00:51.448 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-10T07:00:51.450 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-10T07:00:51.450 DEBUG:teuthology.orchestra.run.smithi174:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-10T07:00:51.509 INFO:tasks.workunit.client.0.smithi174.stderr:++ uuidgen 2024-02-10T07:00:51.516 INFO:tasks.workunit.client.0.smithi174.stderr:+ p=37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:00:51.516 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool create 37577e97-8613-4d15-a71c-97ee4e569923 12 2024-02-10T07:00:51.602 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 -- 172.21.15.174:0/3187293765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 msgr2=0x7fd10c100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:51.602 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 --2- 172.21.15.174:0/3187293765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c100760 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fd108009930 tx=0x7fd10802bc30 comp rx=0 tx=0).stop 2024-02-10T07:00:51.602 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 -- 172.21.15.174:0/3187293765 shutdown_connections 2024-02-10T07:00:51.602 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 --2- 172.21.15.174:0/3187293765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c105450 0x7fd10c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:51.602 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 --2- 172.21.15.174:0/3187293765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c100d30 0x7fd10c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 --2- 172.21.15.174:0/3187293765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c100760 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 -- 172.21.15.174:0/3187293765 >> 172.21.15.174:0/3187293765 conn(0x7fd10c077310 msgr2=0x7fd10c077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.598+0000 7fd113864640 1 -- 172.21.15.174:0/3187293765 shutdown_connections 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 -- 172.21.15.174:0/3187293765 wait complete. 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 Processor -- start 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 -- start start 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c105450 0x7fd10c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c198900 0x7fd10c1a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1a3550 con 0x7fd10c100380 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1a36c0 con 0x7fd10c105450 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd113864640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1a3880 con 0x7fd10c198900 2024-02-10T07:00:51.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd111600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd111600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:51716/0 (socket says 172.21.15.174:51716) 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd111600640 1 -- 172.21.15.174:0/1453829900 learned_addr learned my addr 172.21.15.174:0/1453829900 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd111e01640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c198900 0x7fd10c1a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.599+0000 7fd110dff640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c105450 0x7fd10c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111600640 1 -- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c198900 msgr2=0x7fd10c1a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111600640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c198900 0x7fd10c1a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111600640 1 -- 172.21.15.174:0/1453829900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c105450 msgr2=0x7fd10c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111600640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c105450 0x7fd10c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111600640 1 -- 172.21.15.174:0/1453829900 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108009560 con 0x7fd10c100380 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111e01640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c198900 0x7fd10c1a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd111600640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c197e80 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fd108009230 tx=0x7fd108004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108004440 con 0x7fd10c100380 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fd108003aa0 con 0x7fd10c100380 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c1a3b00 con 0x7fd10c100380 2024-02-10T07:00:51.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108005460 con 0x7fd10c100380 2024-02-10T07:00:51.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.600+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c1a4060 con 0x7fd10c100380 2024-02-10T07:00:51.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.601+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c068560 con 0x7fd10c100380 2024-02-10T07:00:51.606 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.602+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fd1080035f0 con 0x7fd10c100380 2024-02-10T07:00:51.606 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.602+0000 7fd1027fc640 1 --2- 172.21.15.174:0/1453829900 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd0f806d9f0 0x7fd0f806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:51.606 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.602+0000 7fd110dff640 1 --2- 172.21.15.174:0/1453829900 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd0f806d9f0 0x7fd0f806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:51.606 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.602+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(752..752 src has 1..752) v4 ==== 7455+0+0 (secure 0 0 0) 0x7fd1080af800 con 0x7fd10c100380 2024-02-10T07:00:51.606 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.602+0000 7fd110dff640 1 --2- 172.21.15.174:0/1453829900 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd0f806d9f0 0x7fd0f806feb0 secure :-1 s=READY pgs=3438 cs=0 l=1 rev1=1 crypto rx=0x7fd10c06a430 tx=0x7fd0fc00b390 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:00:51.608 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.608+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd10807c8b0 con 0x7fd10c100380 2024-02-10T07:00:51.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:51.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:51 smithi174 ceph-mon[39523]: osdmap e751: 8 total, 8 up, 8 in 2024-02-10T07:00:51.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:51 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:51 smithi174 ceph-mon[29878]: osdmap e751: 8 total, 8 up, 8 in 2024-02-10T07:00:51.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:51 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:51.783 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:51.781+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12} v 0) v1 -- 0x7fd10c100d30 con 0x7fd10c100380 2024-02-10T07:00:51.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:51 smithi184 ceph-mon[33513]: osdmap e751: 8 total, 8 up, 8 in 2024-02-10T07:00:51.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:51 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]: dispatch 2024-02-10T07:00:52.393 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.392+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]=0 pool '37577e97-8613-4d15-a71c-97ee4e569923' created v753) v1 ==== 176+0+0 (secure 0 0 0) 0x7fd10807c440 con 0x7fd10c100380 2024-02-10T07:00:52.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.504+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12} v 0) v1 -- 0x7fd10c10f950 con 0x7fd10c100380 2024-02-10T07:00:52.506 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.504+0000 7fd1027fc640 1 -- 172.21.15.174:0/1453829900 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]=0 pool '37577e97-8613-4d15-a71c-97ee4e569923' already exists v753) v1 ==== 183+0+0 (secure 0 0 0) 0x7fd108043020 con 0x7fd10c100380 2024-02-10T07:00:52.506 INFO:tasks.workunit.client.0.smithi174.stderr:pool '37577e97-8613-4d15-a71c-97ee4e569923' already exists 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd0f806d9f0 msgr2=0x7fd0f806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 --2- 172.21.15.174:0/1453829900 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd0f806d9f0 0x7fd0f806feb0 secure :-1 s=READY pgs=3438 cs=0 l=1 rev1=1 crypto rx=0x7fd10c06a430 tx=0x7fd0fc00b390 comp rx=0 tx=0).stop 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 msgr2=0x7fd10c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c197e80 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fd108009230 tx=0x7fd108004040 comp rx=0 tx=0).stop 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 shutdown_connections 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 --2- 172.21.15.174:0/1453829900 >> v2:172.21.15.174:6800/1252702393 conn(0x7fd0f806d9f0 0x7fd0f806feb0 unknown :-1 s=CLOSED pgs=3438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd10c198900 0x7fd10c1a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c105450 0x7fd10c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 --2- 172.21.15.174:0/1453829900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd10c100380 0x7fd10c197e80 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.508 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 >> 172.21.15.174:0/1453829900 conn(0x7fd10c077310 msgr2=0x7fd10c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:00:52.509 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 shutdown_connections 2024-02-10T07:00:52.509 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.506+0000 7fd113864640 1 -- 172.21.15.174:0/1453829900 wait complete. 2024-02-10T07:00:52.517 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_objects 10 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 -- 172.21.15.174:0/706186045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c101e20 msgr2=0x7f871c105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 --2- 172.21.15.174:0/706186045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c101e20 0x7f871c105e60 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f870c009900 tx=0x7f870c02bc30 comp rx=0 tx=0).stop 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 -- 172.21.15.174:0/706186045 shutdown_connections 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 --2- 172.21.15.174:0/706186045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871c106540 0x7f871c10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 --2- 172.21.15.174:0/706186045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c101e20 0x7f871c105e60 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 --2- 172.21.15.174:0/706186045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 -- 172.21.15.174:0/706186045 >> 172.21.15.174:0/706186045 conn(0x7f871c0fccf0 msgr2=0x7f871c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 -- 172.21.15.174:0/706186045 shutdown_connections 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.598+0000 7f872431e640 1 -- 172.21.15.174:0/706186045 wait complete. 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 Processor -- start 2024-02-10T07:00:52.603 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 -- start start 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871c106540 0x7f871c197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c19b8b0 0x7f871c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c19c2a0 con 0x7f871c101470 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c19c410 con 0x7f871c106540 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c19c580 con 0x7f871c19b8b0 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87228bb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c19b8b0 0x7f871c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:51740/0 (socket says 172.21.15.174:51740) 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 -- 172.21.15.174:0/208461381 learned_addr learned my addr 172.21.15.174:0/208461381 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87218b9640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871c106540 0x7f871c197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 -- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c19b8b0 msgr2=0x7f871c19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:52.604 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c19b8b0 0x7f871c19bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 -- 172.21.15.174:0/208461381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871c106540 msgr2=0x7f871c197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871c106540 0x7f871c197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 -- 172.21.15.174:0/208461381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8718009490 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87228bb640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c19b8b0 0x7f871c19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87220ba640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c196ff0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f871800ea60 tx=0x7f871800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871800cde0 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.599+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c009560 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.600+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f87180044d0 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.600+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718010690 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.600+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c1a3ea0 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.600+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c068560 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.601+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f8718003e10 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.601+0000 7f87137fe640 1 --2- 172.21.15.174:0/208461381 >> v2:172.21.15.174:6800/1252702393 conn(0x7f870806d9f0 0x7f870806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.601+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(753..753 src has 1..753) v4 ==== 7830+0+0 (secure 0 0 0) 0x7f8718014070 con 0x7f871c101470 2024-02-10T07:00:52.605 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.601+0000 7f87218b9640 1 --2- 172.21.15.174:0/208461381 >> v2:172.21.15.174:6800/1252702393 conn(0x7f870806d9f0 0x7f870806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:52.606 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.602+0000 7f87218b9640 1 --2- 172.21.15.174:0/208461381 >> v2:172.21.15.174:6800/1252702393 conn(0x7f870806d9f0 0x7f870806feb0 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 crypto rx=0x7f870c009230 tx=0x7f870c005fd0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:00:52.608 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.607+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8718093020 con 0x7f871c101470 2024-02-10T07:00:52.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[29878]: pgmap v1166: 164 pgs: 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[29878]: osdmap e752: 8 total, 8 up, 8 in 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]: dispatch 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[39523]: pgmap v1166: 164 pgs: 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:52.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[39523]: osdmap e752: 8 total, 8 up, 8 in 2024-02-10T07:00:52.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]: dispatch 2024-02-10T07:00:52.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:52.783 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:52.783+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f871c101e20 con 0x7f871c101470 2024-02-10T07:00:52.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:52 smithi184 ceph-mon[33513]: pgmap v1166: 164 pgs: 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:52.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:52 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:52.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1717407538' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi174-79953-111"}]': finished 2024-02-10T07:00:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:52 smithi184 ceph-mon[33513]: osdmap e752: 8 total, 8 up, 8 in 2024-02-10T07:00:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:52 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]: dispatch 2024-02-10T07:00:52.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:00:53.401 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:53.401+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v754) v1 ==== 223+0+0 (secure 0 0 0) 0x7f871805acc0 con 0x7f871c101470 2024-02-10T07:00:53.514 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:53.513+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f871c102030 con 0x7f871c101470 2024-02-10T07:00:53.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]': finished 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[29878]: osdmap e753: 8 total, 8 up, 8 in 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]: dispatch 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]': finished 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[39523]: osdmap e753: 8 total, 8 up, 8 in 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]: dispatch 2024-02-10T07:00:53.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:53 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:00:53.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]': finished 2024-02-10T07:00:53.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:53 smithi184 ceph-mon[33513]: osdmap e753: 8 total, 8 up, 8 in 2024-02-10T07:00:53.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1453829900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pg_num": 12}]: dispatch 2024-02-10T07:00:53.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:53 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:00:54.403 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.403+0000 7f87137fe640 1 -- 172.21.15.174:0/208461381 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v755) v1 ==== 223+0+0 (secure 0 0 0) 0x7f871805a850 con 0x7f871c101470 2024-02-10T07:00:54.403 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_objects = 10 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 >> v2:172.21.15.174:6800/1252702393 conn(0x7f870806d9f0 msgr2=0x7f870806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 --2- 172.21.15.174:0/208461381 >> v2:172.21.15.174:6800/1252702393 conn(0x7f870806d9f0 0x7f870806feb0 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 crypto rx=0x7f870c009230 tx=0x7f870c005fd0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 msgr2=0x7f871c196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c196ff0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f871800ea60 tx=0x7f871800ef20 comp rx=0 tx=0).stop 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 shutdown_connections 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 --2- 172.21.15.174:0/208461381 >> v2:172.21.15.174:6800/1252702393 conn(0x7f870806d9f0 0x7f870806feb0 unknown :-1 s=CLOSED pgs=3439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f871c19b8b0 0x7f871c19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f871c106540 0x7f871c197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 --2- 172.21.15.174:0/208461381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f871c101470 0x7f871c196ff0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 >> 172.21.15.174:0/208461381 conn(0x7f871c0fccf0 msgr2=0x7f871c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 shutdown_connections 2024-02-10T07:00:54.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.406+0000 7f872431e640 1 -- 172.21.15.174:0/208461381 wait complete. 2024-02-10T07:00:54.421 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool application enable 37577e97-8613-4d15-a71c-97ee4e569923 rados 2024-02-10T07:00:54.515 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 -- 172.21.15.174:0/1653115983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100940 msgr2=0x7f231810d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:54.515 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 --2- 172.21.15.174:0/1653115983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100940 0x7f231810d690 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2308009900 tx=0x7f230802bc30 comp rx=0 tx=0).stop 2024-02-10T07:00:54.515 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 -- 172.21.15.174:0/1653115983 shutdown_connections 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 --2- 172.21.15.174:0/1653115983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f231810dbd0 0x7f231810ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 --2- 172.21.15.174:0/1653115983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100940 0x7f231810d690 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 --2- 172.21.15.174:0/1653115983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2318100020 0x7f2318100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 -- 172.21.15.174:0/1653115983 >> 172.21.15.174:0/1653115983 conn(0x7f2318077250 msgr2=0x7f2318077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 -- 172.21.15.174:0/1653115983 shutdown_connections 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 -- 172.21.15.174:0/1653115983 wait complete. 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 Processor -- start 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 -- start start 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 0x7f2318199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.513+0000 7f231f6c0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2318100940 0x7f23181996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231f6c0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231810dbd0 0x7f231819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231f6c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231819e040 con 0x7f2318100020 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231f6c0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231819e1b0 con 0x7f231810dbd0 2024-02-10T07:00:54.516 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231f6c0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231819e370 con 0x7f2318100940 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231d45c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 0x7f2318199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2318100940 0x7f23181996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231d45c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 0x7f2318199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:51768/0 (socket says 172.21.15.174:51768) 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231d45c640 1 -- 172.21.15.174:0/295504244 learned_addr learned my addr 172.21.15.174:0/295504244 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231dc5d640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231810dbd0 0x7f231819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 -- 172.21.15.174:0/295504244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231810dbd0 msgr2=0x7f231819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231810dbd0 0x7f231819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 -- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 msgr2=0x7f2318199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 0x7f2318199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 -- 172.21.15.174:0/295504244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314009490 con 0x7f2318100940 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231d45c640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 0x7f2318199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231cc5b640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2318100940 0x7f23181996c0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f2308036040 tx=0x7f2308004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308004aa0 con 0x7f2318100940 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2308003aa0 con 0x7f2318100940 2024-02-10T07:00:54.517 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2308009560 con 0x7f2318100940 2024-02-10T07:00:54.518 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308005500 con 0x7f2318100940 2024-02-10T07:00:54.518 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.514+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23181a5cf0 con 0x7f2318100940 2024-02-10T07:00:54.518 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.515+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2318068110 con 0x7f2318100940 2024-02-10T07:00:54.519 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.516+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f23080035f0 con 0x7f2318100940 2024-02-10T07:00:54.519 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.516+0000 7f230e7fc640 1 --2- 172.21.15.174:0/295504244 >> v2:172.21.15.174:6800/1252702393 conn(0x7f230406d9f0 0x7f230406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:00:54.519 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.516+0000 7f231d45c640 1 --2- 172.21.15.174:0/295504244 >> v2:172.21.15.174:6800/1252702393 conn(0x7f230406d9f0 0x7f230406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:00:54.519 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.516+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(755..755 src has 1..755) v4 ==== 7830+0+0 (secure 0 0 0) 0x7f23080afd30 con 0x7f2318100940 2024-02-10T07:00:54.519 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.517+0000 7f231d45c640 1 --2- 172.21.15.174:0/295504244 >> v2:172.21.15.174:6800/1252702393 conn(0x7f230406d9f0 0x7f230406feb0 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 crypto rx=0x7f2310009fd0 tx=0x7f2310009720 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:00:54.521 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.521+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f230807cbe0 con 0x7f2318100940 2024-02-10T07:00:54.698 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:54.697+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"} v 0) v1 -- 0x7f2318112bf0 con 0x7f2318100940 2024-02-10T07:00:54.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[29878]: pgmap v1169: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:54.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:00:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[29878]: osdmap e754: 8 total, 8 up, 8 in 2024-02-10T07:00:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:00:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[39523]: pgmap v1169: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:00:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[39523]: osdmap e754: 8 total, 8 up, 8 in 2024-02-10T07:00:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:54 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:00:54.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:54 smithi184 ceph-mon[33513]: pgmap v1169: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:54.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:00:54.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:54 smithi184 ceph-mon[33513]: osdmap e754: 8 total, 8 up, 8 in 2024-02-10T07:00:54.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:54 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:00:55.426 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:55.425+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]=0 enabled application 'rados' on pool '37577e97-8613-4d15-a71c-97ee4e569923' v756) v1 ==== 213+0+0 (secure 0 0 0) 0x7f230807c770 con 0x7f2318100940 2024-02-10T07:00:55.532 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:55.531+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"} v 0) v1 -- 0x7f23181a5e80 con 0x7f2318100940 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[39523]: osdmap e755: 8 total, 8 up, 8 in 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/295504244' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[29878]: osdmap e755: 8 total, 8 up, 8 in 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/295504244' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:55.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:55 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:55.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/208461381' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:00:55.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:55 smithi184 ceph-mon[33513]: osdmap e755: 8 total, 8 up, 8 in 2024-02-10T07:00:55.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:55 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/295504244' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:55.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:55 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.428 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.428+0000 7f230e7fc640 1 -- 172.21.15.174:0/295504244 <== mon.2 v2:172.21.15.174:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]=0 enabled application 'rados' on pool '37577e97-8613-4d15-a71c-97ee4e569923' v757) v1 ==== 213+0+0 (secure 0 0 0) 0x7f2308043020 con 0x7f2318100940 2024-02-10T07:00:56.429 INFO:tasks.workunit.client.0.smithi174.stderr:enabled application 'rados' on pool '37577e97-8613-4d15-a71c-97ee4e569923' 2024-02-10T07:00:56.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.431+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 >> v2:172.21.15.174:6800/1252702393 conn(0x7f230406d9f0 msgr2=0x7f230406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:56.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.431+0000 7f231f6c0640 1 --2- 172.21.15.174:0/295504244 >> v2:172.21.15.174:6800/1252702393 conn(0x7f230406d9f0 0x7f230406feb0 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 crypto rx=0x7f2310009fd0 tx=0x7f2310009720 comp rx=0 tx=0).stop 2024-02-10T07:00:56.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2318100940 msgr2=0x7f23181996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:00:56.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2318100940 0x7f23181996c0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f2308036040 tx=0x7f2308004270 comp rx=0 tx=0).stop 2024-02-10T07:00:56.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 shutdown_connections 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 --2- 172.21.15.174:0/295504244 >> v2:172.21.15.174:6800/1252702393 conn(0x7f230406d9f0 0x7f230406feb0 unknown :-1 s=CLOSED pgs=3440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231810dbd0 0x7f231819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2318100940 0x7f23181996c0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 --2- 172.21.15.174:0/295504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2318100020 0x7f2318199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 >> 172.21.15.174:0/295504244 conn(0x7f2318077250 msgr2=0x7f23180ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 shutdown_connections 2024-02-10T07:00:56.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:00:56.432+0000 7f231f6c0640 1 -- 172.21.15.174:0/295504244 wait complete. 2024-02-10T07:00:56.444 INFO:tasks.workunit.client.0.smithi174.stderr:++ seq 1 10 2024-02-10T07:00:56.453 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.453 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj1 /etc/passwd 2024-02-10T07:00:56.493 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.493 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj2 /etc/passwd 2024-02-10T07:00:56.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj3 /etc/passwd 2024-02-10T07:00:56.565 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.565 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj4 /etc/passwd 2024-02-10T07:00:56.600 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.600 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj5 /etc/passwd 2024-02-10T07:00:56.634 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.634 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj6 /etc/passwd 2024-02-10T07:00:56.669 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.669 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj7 /etc/passwd 2024-02-10T07:00:56.703 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.703 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj8 /etc/passwd 2024-02-10T07:00:56.718 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:56 smithi184 ceph-mon[33513]: pgmap v1172: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]': finished 2024-02-10T07:00:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:56 smithi184 ceph-mon[33513]: osdmap e756: 8 total, 8 up, 8 in 2024-02-10T07:00:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:56 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/295504244' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.719 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:56 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.738 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.738 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj9 /etc/passwd 2024-02-10T07:00:56.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[29878]: pgmap v1172: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:56.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]': finished 2024-02-10T07:00:56.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[29878]: osdmap e756: 8 total, 8 up, 8 in 2024-02-10T07:00:56.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/295504244' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[39523]: pgmap v1172: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:00:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]': finished 2024-02-10T07:00:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[39523]: osdmap e756: 8 total, 8 up, 8 in 2024-02-10T07:00:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/295504244' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:56 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]: dispatch 2024-02-10T07:00:56.772 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:00:56.772 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put obj10 /etc/passwd 2024-02-10T07:00:56.806 INFO:tasks.workunit.client.0.smithi174.stderr:+ sleep 30 2024-02-10T07:00:57.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:00:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:00:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]': finished 2024-02-10T07:00:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-mon[39523]: osdmap e757: 8 total, 8 up, 8 in 2024-02-10T07:00:57.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]': finished 2024-02-10T07:00:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-mon[29878]: osdmap e757: 8 total, 8 up, 8 in 2024-02-10T07:00:57.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:57.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:57 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "app": "rados"}]': finished 2024-02-10T07:00:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:57 smithi184 ceph-mon[33513]: osdmap e757: 8 total, 8 up, 8 in 2024-02-10T07:00:57.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:00:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:00:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:00:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:00:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:58 smithi174 ceph-mon[29878]: pgmap v1175: 176 pgs: 176 active+clean; 582 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:58.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:00:58 smithi174 ceph-mon[29878]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:58 smithi174 ceph-mon[39523]: pgmap v1175: 176 pgs: 176 active+clean; 582 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:58.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:00:58 smithi174 ceph-mon[39523]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:00:58.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:58 smithi184 ceph-mon[33513]: pgmap v1175: 176 pgs: 176 active+clean; 582 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:00:58.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:00:58 smithi184 ceph-mon[33513]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:01:00.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:00 smithi174 ceph-mon[29878]: pgmap v1176: 176 pgs: 176 active+clean; 582 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 885 B/s rd, 0 op/s 2024-02-10T07:01:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:00 smithi174 ceph-mon[39523]: pgmap v1176: 176 pgs: 176 active+clean; 582 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 885 B/s rd, 0 op/s 2024-02-10T07:01:00.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:00 smithi184 ceph-mon[33513]: pgmap v1176: 176 pgs: 176 active+clean; 582 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 885 B/s rd, 0 op/s 2024-02-10T07:01:02.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:02 smithi174 ceph-mon[29878]: pgmap v1177: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 KiB/s wr, 2 op/s 2024-02-10T07:01:02.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:02 smithi174 ceph-mon[39523]: pgmap v1177: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 KiB/s wr, 2 op/s 2024-02-10T07:01:02.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:02 smithi184 ceph-mon[33513]: pgmap v1177: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 KiB/s wr, 2 op/s 2024-02-10T07:01:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:04 smithi174 ceph-mon[29878]: pgmap v1178: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T07:01:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:04 smithi174 ceph-mon[39523]: pgmap v1178: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T07:01:04.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:04 smithi184 ceph-mon[33513]: pgmap v1178: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T07:01:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:05 smithi174 ceph-mon[29878]: pool '37577e97-8613-4d15-a71c-97ee4e569923' is full (reached quota's max_objects: 10) 2024-02-10T07:01:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:05 smithi174 ceph-mon[29878]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:01:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:05 smithi174 ceph-mon[29878]: osdmap e758: 8 total, 8 up, 8 in 2024-02-10T07:01:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:05 smithi174 ceph-mon[39523]: pool '37577e97-8613-4d15-a71c-97ee4e569923' is full (reached quota's max_objects: 10) 2024-02-10T07:01:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:05 smithi174 ceph-mon[39523]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:01:05.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:05 smithi174 ceph-mon[39523]: osdmap e758: 8 total, 8 up, 8 in 2024-02-10T07:01:05.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:05 smithi184 ceph-mon[33513]: pool '37577e97-8613-4d15-a71c-97ee4e569923' is full (reached quota's max_objects: 10) 2024-02-10T07:01:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:05 smithi184 ceph-mon[33513]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:01:05.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:05 smithi184 ceph-mon[33513]: osdmap e758: 8 total, 8 up, 8 in 2024-02-10T07:01:06.730 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:06 smithi184 ceph-mon[33513]: pgmap v1180: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 584 B/s rd, 14 KiB/s wr, 1 op/s 2024-02-10T07:01:06.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:06 smithi174 ceph-mon[29878]: pgmap v1180: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 584 B/s rd, 14 KiB/s wr, 1 op/s 2024-02-10T07:01:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:06 smithi174 ceph-mon[39523]: pgmap v1180: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 584 B/s rd, 14 KiB/s wr, 1 op/s 2024-02-10T07:01:07.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:01:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:01:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:01:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:01:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:01:07.873 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:01:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:01:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:01:08.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:08 smithi174 ceph-mon[29878]: pgmap v1181: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:01:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:08 smithi174 ceph-mon[39523]: pgmap v1181: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:01:08.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:08 smithi184 ceph-mon[33513]: pgmap v1181: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:01:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:10 smithi174 ceph-mon[29878]: pgmap v1182: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:01:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:10 smithi174 ceph-mon[39523]: pgmap v1182: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:01:10.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:10 smithi184 ceph-mon[33513]: pgmap v1182: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:01:12.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:12 smithi174 ceph-mon[29878]: pgmap v1183: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:12 smithi174 ceph-mon[39523]: pgmap v1183: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:12.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:12 smithi184 ceph-mon[33513]: pgmap v1183: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:14 smithi174 ceph-mon[29878]: pgmap v1184: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:14 smithi174 ceph-mon[39523]: pgmap v1184: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:14.872 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:14 smithi184 ceph-mon[33513]: pgmap v1184: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:16.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:16 smithi184 ceph-mon[33513]: pgmap v1185: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-10T07:01:16.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:16 smithi174 ceph-mon[29878]: pgmap v1185: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-10T07:01:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:16 smithi174 ceph-mon[39523]: pgmap v1185: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-10T07:01:17.004 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:01:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:01:17.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:17.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:18.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:18 smithi174 ceph-mon[29878]: pgmap v1186: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:18 smithi174 ceph-mon[39523]: pgmap v1186: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:01:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:01:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:01:18.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:18 smithi184 ceph-mon[33513]: pgmap v1186: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:20.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:20 smithi184 ceph-mon[33513]: pgmap v1187: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:20 smithi174 ceph-mon[29878]: pgmap v1187: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:20.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:20 smithi174 ceph-mon[39523]: pgmap v1187: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:22 smithi184 ceph-mon[33513]: pgmap v1188: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:22.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:22 smithi174 ceph-mon[29878]: pgmap v1188: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:22 smithi174 ceph-mon[39523]: pgmap v1188: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:24.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:24 smithi184 ceph-mon[33513]: pgmap v1189: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:24.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:24 smithi174 ceph-mon[29878]: pgmap v1189: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:24.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:24 smithi174 ceph-mon[39523]: pgmap v1189: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:26 smithi184 ceph-mon[33513]: pgmap v1190: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:26.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:26 smithi174 ceph-mon[29878]: pgmap v1190: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:26 smithi174 ceph-mon[39523]: pgmap v1190: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=150782 2024-02-10T07:01:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_objects 100 2024-02-10T07:01:26.808 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put onemore /etc/passwd 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/1821038715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58100380 msgr2=0x7f9d58100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/1821038715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58100380 0x7f9d58100760 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009930 tx=0x7f9d5402bc30 comp rx=0 tx=0).stop 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/1821038715 shutdown_connections 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/1821038715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58105450 0x7f9d5810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/1821038715 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58100d30 0x7f9d58104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/1821038715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58100380 0x7f9d58100760 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:26.905 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/1821038715 >> 172.21.15.174:0/1821038715 conn(0x7f9d58077310 msgr2=0x7f9d58077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/1821038715 shutdown_connections 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/1821038715 wait complete. 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 Processor -- start 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.902+0000 7f9d5f4ff640 1 -- start start 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58100380 0x7f9d58101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 0x7f9d58102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58102890 0x7f9d581a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581a5740 con 0x7f9d58105450 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581a58b0 con 0x7f9d58100380 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581a5a70 con 0x7f9d58102890 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 0x7f9d58102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5da9c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58102890 0x7f9d581a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:26.906 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 0x7f9d58102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47098/0 (socket says 172.21.15.174:47098) 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 -- 172.21.15.174:0/3673030957 learned_addr learned my addr 172.21.15.174:0/3673030957 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5d29b640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58100380 0x7f9d58101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 -- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58102890 msgr2=0x7f9d581a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58102890 0x7f9d581a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 -- 172.21.15.174:0/3673030957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58100380 msgr2=0x7f9d58101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58100380 0x7f9d58101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48009490 con 0x7f9d58105450 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5d29b640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58100380 0x7f9d58101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5da9c640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58102890 0x7f9d581a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5ca9a640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 0x7f9d58102350 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9a0 tx=0x7f9d4800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800cd80 con 0x7f9d58105450 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f9d480044d0 con 0x7f9d58105450 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54009560 con 0x7f9d58105450 2024-02-10T07:01:26.907 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48010690 con 0x7f9d58105450 2024-02-10T07:01:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.903+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d581a60d0 con 0x7f9d58105450 2024-02-10T07:01:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.904+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d58068560 con 0x7f9d58105450 2024-02-10T07:01:26.908 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.905+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f9d48003e10 con 0x7f9d58105450 2024-02-10T07:01:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.905+0000 7f9d4e7fc640 1 --2- 172.21.15.174:0/3673030957 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406d9f0 0x7f9d4406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.905+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(758..758 src has 1..758) v4 ==== 7843+0+0 (secure 0 0 0) 0x7f9d48014070 con 0x7f9d58105450 2024-02-10T07:01:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.905+0000 7f9d5d29b640 1 --2- 172.21.15.174:0/3673030957 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406d9f0 0x7f9d4406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.906+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=759}) v3 -- 0x7f9d44078fc0 con 0x7f9d58105450 2024-02-10T07:01:26.909 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.906+0000 7f9d5d29b640 1 --2- 172.21.15.174:0/3673030957 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406d9f0 0x7f9d4406feb0 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005ec0 tx=0x7f9d54000bc0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:01:26.910 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:26.910+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d4805acb0 con 0x7f9d58105450 2024-02-10T07:01:27.008 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:01:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:01:27.085 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:27.085+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f9d58100d30 con 0x7f9d58105450 2024-02-10T07:01:27.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:27.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:28.013 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:28.013+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v759) v1 ==== 225+0+0 (secure 0 0 0) 0x7f9d4805a840 con 0x7f9d58105450 2024-02-10T07:01:28.026 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:28.026+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 8 ==== osd_map(759..759 src has 1..759) v4 ==== 628+0+0 (secure 0 0 0) 0x7f9d48003780 con 0x7f9d58105450 2024-02-10T07:01:28.026 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:28.026+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=760}) v3 -- 0x7f9d44079a80 con 0x7f9d58105450 2024-02-10T07:01:28.124 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:28.123+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f9d58100ec0 con 0x7f9d58105450 2024-02-10T07:01:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:01:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:01:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:01:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:28 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T07:01:28.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:28 smithi174 ceph-mon[29878]: pgmap v1191: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:28 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T07:01:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:28 smithi174 ceph-mon[39523]: pgmap v1191: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:28 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T07:01:28.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:28 smithi184 ceph-mon[33513]: pgmap v1191: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:29.018 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.017+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v760) v1 ==== 225+0+0 (secure 0 0 0) 0x7f9d4805aa20 con 0x7f9d58105450 2024-02-10T07:01:29.018 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_objects = 100 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d4e7fc640 1 -- 172.21.15.174:0/3673030957 <== mon.0 v2:172.21.15.174:3300/0 10 ==== osd_map(760..760 src has 1..760) v4 ==== 628+0+0 (secure 0 0 0) 0x7f9d4805b5e0 con 0x7f9d58105450 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406d9f0 msgr2=0x7f9d4406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/3673030957 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406d9f0 0x7f9d4406feb0 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f9d54005ec0 tx=0x7f9d54000bc0 comp rx=0 tx=0).stop 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 msgr2=0x7f9d58102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 0x7f9d58102350 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9a0 tx=0x7f9d4800ee60 comp rx=0 tx=0).stop 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 shutdown_connections 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/3673030957 >> v2:172.21.15.174:6800/1252702393 conn(0x7f9d4406d9f0 0x7f9d4406feb0 unknown :-1 s=CLOSED pgs=3452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d58102890 0x7f9d581a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:29.021 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d58105450 0x7f9d58102350 secure :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9a0 tx=0x7f9d4800ee60 comp rx=0 tx=0).stop 2024-02-10T07:01:29.022 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 --2- 172.21.15.174:0/3673030957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d58100380 0x7f9d58101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:29.022 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 >> 172.21.15.174:0/3673030957 conn(0x7f9d58077310 msgr2=0x7f9d58103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:01:29.022 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 shutdown_connections 2024-02-10T07:01:29.022 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:29.020+0000 7f9d5f4ff640 1 -- 172.21.15.174:0/3673030957 wait complete. 2024-02-10T07:01:29.035 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 150782 2024-02-10T07:01:29.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]': finished 2024-02-10T07:01:29.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:29 smithi184 ceph-mon[33513]: osdmap e759: 8 total, 8 up, 8 in 2024-02-10T07:01:29.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:29 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T07:01:29.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]': finished 2024-02-10T07:01:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:29 smithi174 ceph-mon[29878]: osdmap e759: 8 total, 8 up, 8 in 2024-02-10T07:01:29.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:29 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T07:01:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]': finished 2024-02-10T07:01:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:29 smithi174 ceph-mon[39523]: osdmap e759: 8 total, 8 up, 8 in 2024-02-10T07:01:29.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:29 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T07:01:30.016 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 0 -ne 0 ']' 2024-02-10T07:01:30.016 INFO:tasks.workunit.client.0.smithi174.stderr:+ true 2024-02-10T07:01:30.016 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put twomore /etc/passwd 2024-02-10T07:01:30.056 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_bytes 100 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 -- 172.21.15.174:0/1255507418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce8100e70 msgr2=0x7f1ce8101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/1255507418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce8100e70 0x7f1ce8101250 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009900 tx=0x7f1ce402bc30 comp rx=0 tx=0).stop 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 -- 172.21.15.174:0/1255507418 shutdown_connections 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/1255507418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce81054f0 0x7f1ce810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/1255507418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8101820 0x7f1ce8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/1255507418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce8100e70 0x7f1ce8101250 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 -- 172.21.15.174:0/1255507418 >> 172.21.15.174:0/1255507418 conn(0x7f1ce8075450 msgr2=0x7f1ce8075860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 -- 172.21.15.174:0/1255507418 shutdown_connections 2024-02-10T07:01:30.144 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.141+0000 7f1cf07e9640 1 -- 172.21.15.174:0/1255507418 wait complete. 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 Processor -- start 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 -- start start 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 0x7f1ce8196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce81054f0 0x7f1ce8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 0x7f1ce819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce819c2a0 con 0x7f1ce8100e70 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce819c410 con 0x7f1ce81054f0 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cf07e9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce819c580 con 0x7f1ce819b8b0 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1ceed86640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 0x7f1ce819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 0x7f1ce8196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 0x7f1ce8196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47134/0 (socket says 172.21.15.174:47134) 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 -- 172.21.15.174:0/226727769 learned_addr learned my addr 172.21.15.174:0/226727769 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1ceed86640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 0x7f1ce819bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:47506/0 (socket says 172.21.15.174:47506) 2024-02-10T07:01:30.145 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cedd84640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce81054f0 0x7f1ce8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 -- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 msgr2=0x7f1ce819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 0x7f1ce819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 -- 172.21.15.174:0/226727769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce81054f0 msgr2=0x7f1ce8197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce81054f0 0x7f1ce8197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1cee585640 1 -- 172.21.15.174:0/226727769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8009490 con 0x7f1ce8100e70 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.142+0000 7f1ceed86640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 0x7f1ce819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.143+0000 7f1cee585640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 0x7f1ce8196ff0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005940 tx=0x7f1ce40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.143+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40044c0 con 0x7f1ce8100e70 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.143+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce4009560 con 0x7f1ce8100e70 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.143+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1ce4003aa0 con 0x7f1ce8100e70 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.143+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4005500 con 0x7f1ce8100e70 2024-02-10T07:01:30.146 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.143+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce81a3e50 con 0x7f1ce8100e70 2024-02-10T07:01:30.147 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.144+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce8069540 con 0x7f1ce8100e70 2024-02-10T07:01:30.147 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.144+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f1ce40035f0 con 0x7f1ce8100e70 2024-02-10T07:01:30.147 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.145+0000 7f1cdf7fe640 1 --2- 172.21.15.174:0/226727769 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1cd406d9f0 0x7f1cd406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:01:30.147 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.145+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(761..761 src has 1..761) v4 ==== 7843+0+0 (secure 0 0 0) 0x7f1ce40afc50 con 0x7f1ce8100e70 2024-02-10T07:01:30.148 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.145+0000 7f1cedd84640 1 --2- 172.21.15.174:0/226727769 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1cd406d9f0 0x7f1cd406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:01:30.148 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.145+0000 7f1cedd84640 1 --2- 172.21.15.174:0/226727769 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1cd406d9f0 0x7f1cd406feb0 secure :-1 s=READY pgs=3454 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80fd0b0 tx=0x7f1cd8005f90 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:01:30.149 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.149+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ce40b5020 con 0x7f1ce8100e70 2024-02-10T07:01:30.321 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:30.321+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f1ce8198340 con 0x7f1ce8100e70 2024-02-10T07:01:30.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:30 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]': finished 2024-02-10T07:01:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:30 smithi184 ceph-mon[33513]: osdmap e760: 8 total, 8 up, 8 in 2024-02-10T07:01:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:30 smithi184 ceph-mon[33513]: pgmap v1194: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:01:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:30 smithi184 ceph-mon[33513]: pool '37577e97-8613-4d15-a71c-97ee4e569923' no longer out of quota; removing NO_QUOTA flag 2024-02-10T07:01:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:30 smithi184 ceph-mon[33513]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:01:30.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:30 smithi184 ceph-mon[33513]: osdmap e761: 8 total, 8 up, 8 in 2024-02-10T07:01:30.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]': finished 2024-02-10T07:01:30.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[29878]: osdmap e760: 8 total, 8 up, 8 in 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[29878]: pgmap v1194: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[29878]: pool '37577e97-8613-4d15-a71c-97ee4e569923' no longer out of quota; removing NO_QUOTA flag 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[29878]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[29878]: osdmap e761: 8 total, 8 up, 8 in 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3673030957' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "100"}]': finished 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[39523]: osdmap e760: 8 total, 8 up, 8 in 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[39523]: pgmap v1194: 176 pgs: 176 active+clean; 697 KiB data, 816 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[39523]: pool '37577e97-8613-4d15-a71c-97ee4e569923' no longer out of quota; removing NO_QUOTA flag 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[39523]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:01:30.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:30 smithi174 ceph-mon[39523]: osdmap e761: 8 total, 8 up, 8 in 2024-02-10T07:01:31.043 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:31.042+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v762) v1 ==== 221+0+0 (secure 0 0 0) 0x7f1ce407caf0 con 0x7f1ce8100e70 2024-02-10T07:01:31.153 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:31.153+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f1ce8101a00 con 0x7f1ce8100e70 2024-02-10T07:01:31.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:31 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T07:01:31.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:31 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T07:01:31.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:31 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T07:01:32.048 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.048+0000 7f1cdf7fe640 1 -- 172.21.15.174:0/226727769 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v763) v1 ==== 221+0+0 (secure 0 0 0) 0x7f1ce407c680 con 0x7f1ce8100e70 2024-02-10T07:01:32.048 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_bytes = 100 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:01:32.051 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.051+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1cd406d9f0 msgr2=0x7f1cd406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:32.052 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.051+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/226727769 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1cd406d9f0 0x7f1cd406feb0 secure :-1 s=READY pgs=3454 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80fd0b0 tx=0x7f1cd8005f90 comp rx=0 tx=0).stop 2024-02-10T07:01:32.052 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.051+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 msgr2=0x7f1ce8196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:01:32.052 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.051+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 0x7f1ce8196ff0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005940 tx=0x7f1ce40040c0 comp rx=0 tx=0).stop 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 shutdown_connections 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/226727769 >> v2:172.21.15.174:6800/1252702393 conn(0x7f1cd406d9f0 0x7f1cd406feb0 unknown :-1 s=CLOSED pgs=3454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ce819b8b0 0x7f1ce819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ce81054f0 0x7f1ce8197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 --2- 172.21.15.174:0/226727769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ce8100e70 0x7f1ce8196ff0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 >> 172.21.15.174:0/226727769 conn(0x7f1ce8075450 msgr2=0x7f1ce8104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 shutdown_connections 2024-02-10T07:01:32.054 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:01:32.053+0000 7f1cf07e9640 1 -- 172.21.15.174:0/226727769 wait complete. 2024-02-10T07:01:32.064 INFO:tasks.workunit.client.0.smithi174.stderr:+ sleep 30 2024-02-10T07:01:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T07:01:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:32 smithi184 ceph-mon[33513]: osdmap e762: 8 total, 8 up, 8 in 2024-02-10T07:01:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:32 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T07:01:32.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:32 smithi184 ceph-mon[33513]: pgmap v1197: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T07:01:32.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[29878]: osdmap e762: 8 total, 8 up, 8 in 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[29878]: pgmap v1197: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[39523]: osdmap e762: 8 total, 8 up, 8 in 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T07:01:32.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:32 smithi174 ceph-mon[39523]: pgmap v1197: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T07:01:33.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:33 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T07:01:33.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:33 smithi184 ceph-mon[33513]: osdmap e763: 8 total, 8 up, 8 in 2024-02-10T07:01:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:33 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T07:01:33.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:33 smithi174 ceph-mon[29878]: osdmap e763: 8 total, 8 up, 8 in 2024-02-10T07:01:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:33 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/226727769' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T07:01:33.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:33 smithi174 ceph-mon[39523]: osdmap e763: 8 total, 8 up, 8 in 2024-02-10T07:01:34.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:34 smithi184 ceph-mon[33513]: pgmap v1199: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.8 KiB/s wr, 1 op/s 2024-02-10T07:01:34.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:34 smithi174 ceph-mon[29878]: pgmap v1199: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.8 KiB/s wr, 1 op/s 2024-02-10T07:01:34.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:34 smithi174 ceph-mon[39523]: pgmap v1199: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.8 KiB/s wr, 1 op/s 2024-02-10T07:01:35.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:35 smithi184 ceph-mon[33513]: pool '37577e97-8613-4d15-a71c-97ee4e569923' is full (reached quota's max_bytes: 100 B) 2024-02-10T07:01:35.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:35 smithi184 ceph-mon[33513]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:01:35.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:35 smithi184 ceph-mon[33513]: osdmap e764: 8 total, 8 up, 8 in 2024-02-10T07:01:35.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:35 smithi174 ceph-mon[29878]: pool '37577e97-8613-4d15-a71c-97ee4e569923' is full (reached quota's max_bytes: 100 B) 2024-02-10T07:01:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:35 smithi174 ceph-mon[29878]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:01:35.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:35 smithi174 ceph-mon[29878]: osdmap e764: 8 total, 8 up, 8 in 2024-02-10T07:01:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:35 smithi174 ceph-mon[39523]: pool '37577e97-8613-4d15-a71c-97ee4e569923' is full (reached quota's max_bytes: 100 B) 2024-02-10T07:01:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:35 smithi174 ceph-mon[39523]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:01:35.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:35 smithi174 ceph-mon[39523]: osdmap e764: 8 total, 8 up, 8 in 2024-02-10T07:01:36.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:36 smithi184 ceph-mon[33513]: pgmap v1201: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 986 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-02-10T07:01:36.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:36 smithi174 ceph-mon[29878]: pgmap v1201: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 986 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-02-10T07:01:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:36 smithi174 ceph-mon[39523]: pgmap v1201: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 986 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-02-10T07:01:37.069 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:01:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:01:37.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:37.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:37 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:37 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:37 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:38.073 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:01:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:01:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:01:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:38 smithi184 ceph-mon[33513]: pgmap v1202: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:01:38.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:38 smithi174 ceph-mon[29878]: pgmap v1202: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:01:38.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:38 smithi174 ceph-mon[39523]: pgmap v1202: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:01:40.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:40 smithi184 ceph-mon[33513]: pgmap v1203: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:01:40.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:40 smithi174 ceph-mon[29878]: pgmap v1203: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:01:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:40 smithi174 ceph-mon[39523]: pgmap v1203: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:01:42.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:42 smithi184 ceph-mon[33513]: pgmap v1204: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:01:42.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:42 smithi174 ceph-mon[29878]: pgmap v1204: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:01:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:42 smithi174 ceph-mon[39523]: pgmap v1204: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:01:44.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:44 smithi184 ceph-mon[33513]: pgmap v1205: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:44.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:44 smithi174 ceph-mon[29878]: pgmap v1205: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:44 smithi174 ceph-mon[39523]: pgmap v1205: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:01:46.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:46 smithi184 ceph-mon[33513]: pgmap v1206: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-10T07:01:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:46 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:01:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:46 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:01:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:46 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:01:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:46 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:01:46.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[29878]: pgmap v1206: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[39523]: pgmap v1206: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:01:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:46 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:01:47.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:01:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:01:47.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:47.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:48.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:48 smithi174 ceph-mon[29878]: pgmap v1207: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:01:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:01:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:01:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:48 smithi174 ceph-mon[39523]: pgmap v1207: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:48 smithi184 ceph-mon[33513]: pgmap v1207: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:50.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:50 smithi184 ceph-mon[33513]: pgmap v1208: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:50.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:50 smithi174 ceph-mon[29878]: pgmap v1208: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:50 smithi174 ceph-mon[39523]: pgmap v1208: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:52.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:52 smithi184 ceph-mon[33513]: pgmap v1209: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:52 smithi174 ceph-mon[29878]: pgmap v1209: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:52 smithi174 ceph-mon[39523]: pgmap v1209: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:01:54.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:54 smithi184 ceph-mon[33513]: pgmap v1210: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:54 smithi174 ceph-mon[29878]: pgmap v1210: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:54 smithi174 ceph-mon[39523]: pgmap v1210: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:56.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:56 smithi184 ceph-mon[33513]: pgmap v1211: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:56 smithi174 ceph-mon[29878]: pgmap v1211: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:56 smithi174 ceph-mon[39523]: pgmap v1211: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:01:57.018 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:01:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:01:57.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:57.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:01:58.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:01:58 smithi174 ceph-mon[29878]: pgmap v1212: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:01:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:01:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:01:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:01:58 smithi174 ceph-mon[39523]: pgmap v1212: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:01:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:01:58 smithi184 ceph-mon[33513]: pgmap v1212: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:00.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:00 smithi184 ceph-mon[33513]: pgmap v1213: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:00.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:00 smithi174 ceph-mon[29878]: pgmap v1213: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:00 smithi174 ceph-mon[39523]: pgmap v1213: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:02.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ pid=151000 2024-02-10T07:02:02.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_bytes 0 2024-02-10T07:02:02.066 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put two /etc/passwd 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 -- 172.21.15.174:0/1630229508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf00ff8b0 msgr2=0x7fdaf00ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 --2- 172.21.15.174:0/1630229508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf00ff8b0 0x7fdaf00ffd30 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009930 tx=0x7fdae002bc30 comp rx=0 tx=0).stop 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 -- 172.21.15.174:0/1630229508 shutdown_connections 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 --2- 172.21.15.174:0/1630229508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaf0100270 0x7fdaf010c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 --2- 172.21.15.174:0/1630229508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf00ff8b0 0x7fdaf00ffd30 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 --2- 172.21.15.174:0/1630229508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf0105870 0x7fdaf0105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 -- 172.21.15.174:0/1630229508 >> 172.21.15.174:0/1630229508 conn(0x7fdaf0076620 msgr2=0x7fdaf00fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 -- 172.21.15.174:0/1630229508 shutdown_connections 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.160+0000 7fdaf7664640 1 -- 172.21.15.174:0/1630229508 wait complete. 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 Processor -- start 2024-02-10T07:02:02.164 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 -- start start 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf00ff8b0 0x7fdaf0196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaf0100270 0x7fdaf01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf0105870 0x7fdaf019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf019be10 con 0x7fdaf0105870 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf019bf80 con 0x7fdaf0100270 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf7664640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf019c140 con 0x7fdaf00ff8b0 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5400640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf00ff8b0 0x7fdaf0196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5c01640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf0105870 0x7fdaf019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5400640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf00ff8b0 0x7fdaf0196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:48660/0 (socket says 172.21.15.174:48660) 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5400640 1 -- 172.21.15.174:0/762256297 learned_addr learned my addr 172.21.15.174:0/762256297 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf4bff640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaf0100270 0x7fdaf01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5c01640 1 -- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf00ff8b0 msgr2=0x7fdaf0196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5c01640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf00ff8b0 0x7fdaf0196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5c01640 1 -- 172.21.15.174:0/762256297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaf0100270 msgr2=0x7fdaf01974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:02.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5c01640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaf0100270 0x7fdaf01974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.161+0000 7fdaf5c01640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec009490 con 0x7fdaf0105870 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.162+0000 7fdaf5c01640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf0105870 0x7fdaf019b810 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fdae800ddc0 tx=0x7fdae800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.162+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004d20 con 0x7fdaf0105870 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.162+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fdae8004e80 con 0x7fdaf0105870 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.162+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae0009560 con 0x7fdaf0105870 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.162+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8005430 con 0x7fdaf0105870 2024-02-10T07:02:02.166 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.162+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf01a3ac0 con 0x7fdaf0105870 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.163+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf0100fa0 con 0x7fdaf0105870 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.163+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fdae8005590 con 0x7fdaf0105870 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.164+0000 7fdae67fc640 1 --2- 172.21.15.174:0/762256297 >> v2:172.21.15.174:6800/1252702393 conn(0x7fdadc06d9f0 0x7fdadc06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.164+0000 7fdaf5400640 1 --2- 172.21.15.174:0/762256297 >> v2:172.21.15.174:6800/1252702393 conn(0x7fdadc06d9f0 0x7fdadc06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.164+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(764..764 src has 1..764) v4 ==== 7843+0+0 (secure 0 0 0) 0x7fdae808db80 con 0x7fdaf0105870 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.164+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=765}) v3 -- 0x7fdadc079040 con 0x7fdaf0105870 2024-02-10T07:02:02.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.164+0000 7fdaf5400640 1 --2- 172.21.15.174:0/762256297 >> v2:172.21.15.174:6800/1252702393 conn(0x7fdadc06d9f0 0x7fdadc06feb0 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009de0 tx=0x7fdaec005f90 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:02.169 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.168+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdae805aaa0 con 0x7fdaf0105870 2024-02-10T07:02:02.345 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:02.343+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fdaf01982a0 con 0x7fdaf0105870 2024-02-10T07:02:02.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:02 smithi184 ceph-mon[33513]: pgmap v1214: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:02.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:02 smithi174 ceph-mon[29878]: pgmap v1214: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:02.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:02 smithi174 ceph-mon[39523]: pgmap v1214: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:03.271 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:03.268+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v765) v1 ==== 217+0+0 (secure 0 0 0) 0x7fdae805a630 con 0x7fdaf0105870 2024-02-10T07:02:03.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:03.282+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 8 ==== osd_map(765..765 src has 1..765) v4 ==== 628+0+0 (secure 0 0 0) 0x7fdae805ed70 con 0x7fdaf0105870 2024-02-10T07:02:03.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:03.282+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=766}) v3 -- 0x7fdadc079a80 con 0x7fdaf0105870 2024-02-10T07:02:03.384 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:03.383+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fdaf0109de0 con 0x7fdaf0105870 2024-02-10T07:02:03.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:03 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:02:03.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:03 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:02:03.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:03 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:02:04.280 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.278+0000 7fdae67fc640 1 -- 172.21.15.174:0/762256297 <== mon.0 v2:172.21.15.174:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v766) v1 ==== 217+0+0 (secure 0 0 0) 0x7fdae805a810 con 0x7fdaf0105870 2024-02-10T07:02:04.280 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_bytes = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:02:04.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 >> v2:172.21.15.174:6800/1252702393 conn(0x7fdadc06d9f0 msgr2=0x7fdadc06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:04.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 --2- 172.21.15.174:0/762256297 >> v2:172.21.15.174:6800/1252702393 conn(0x7fdadc06d9f0 0x7fdadc06feb0 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009de0 tx=0x7fdaec005f90 comp rx=0 tx=0).stop 2024-02-10T07:02:04.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf0105870 msgr2=0x7fdaf019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:04.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf0105870 0x7fdaf019b810 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fdae800ddc0 tx=0x7fdae800b730 comp rx=0 tx=0).stop 2024-02-10T07:02:04.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 shutdown_connections 2024-02-10T07:02:04.283 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 --2- 172.21.15.174:0/762256297 >> v2:172.21.15.174:6800/1252702393 conn(0x7fdadc06d9f0 0x7fdadc06feb0 unknown :-1 s=CLOSED pgs=3456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.284 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaf0105870 0x7fdaf019b810 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.284 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.281+0000 7fdaf7664640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaf0100270 0x7fdaf01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.284 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.282+0000 7fdaf7664640 1 --2- 172.21.15.174:0/762256297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdaf00ff8b0 0x7fdaf0196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.284 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.282+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 >> 172.21.15.174:0/762256297 conn(0x7fdaf0076620 msgr2=0x7fdaf00fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:04.284 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.282+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 shutdown_connections 2024-02-10T07:02:04.284 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.282+0000 7fdaf7664640 1 -- 172.21.15.174:0/762256297 wait complete. 2024-02-10T07:02:04.298 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_objects 0 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 -- 172.21.15.174:0/2811822002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 msgr2=0x7fad1010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 --2- 172.21.15.174:0/2811822002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad1010d020 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fad08009cd0 tx=0x7fad0802c0f0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 -- 172.21.15.174:0/2811822002 shutdown_connections 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 --2- 172.21.15.174:0/2811822002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad1010d020 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 --2- 172.21.15.174:0/2811822002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10100d30 0x7fad10104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 --2- 172.21.15.174:0/2811822002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 0x7fad10100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 -- 172.21.15.174:0/2811822002 >> 172.21.15.174:0/2811822002 conn(0x7fad10077310 msgr2=0x7fad10077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 -- 172.21.15.174:0/2811822002 shutdown_connections 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.384+0000 7fad14d7a640 1 -- 172.21.15.174:0/2811822002 wait complete. 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 Processor -- start 2024-02-10T07:02:04.388 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 -- start start 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 0x7fad10197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad101982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10198840 0x7fad101a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad101a3490 con 0x7fad10105450 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad101a3600 con 0x7fad10198840 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad14d7a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad101a37c0 con 0x7fad10100380 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad07fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad101982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 0x7fad10197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad07fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad101982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50268/0 (socket says 172.21.15.174:50268) 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad07fff640 1 -- 172.21.15.174:0/949646328 learned_addr learned my addr 172.21.15.174:0/949646328 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0ed9d640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10198840 0x7fad101a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 -- 172.21.15.174:0/949646328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10198840 msgr2=0x7fad101a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10198840 0x7fad101a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 -- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 msgr2=0x7fad101982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:04.389 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad101982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad00009490 con 0x7fad10100380 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad07fff640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad101982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.385+0000 7fad0e59c640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 0x7fad10197d70 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e9f0 tx=0x7fad0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.386+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000cd80 con 0x7fad10100380 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.386+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fad000044d0 con 0x7fad10100380 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.386+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08009900 con 0x7fad10100380 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.386+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00010430 con 0x7fad10100380 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.386+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad101a3e20 con 0x7fad10100380 2024-02-10T07:02:04.390 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.387+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad10068560 con 0x7fad10100380 2024-02-10T07:02:04.391 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.387+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fad00003f70 con 0x7fad10100380 2024-02-10T07:02:04.391 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.387+0000 7fad077fe640 1 --2- 172.21.15.174:0/949646328 >> v2:172.21.15.174:6800/1252702393 conn(0x7facfc06d9f0 0x7facfc06feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:04.391 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.387+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(766..766 src has 1..766) v4 ==== 7843+0+0 (secure 0 0 0) 0x7fad00014070 con 0x7fad10100380 2024-02-10T07:02:04.391 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.387+0000 7fad07fff640 1 --2- 172.21.15.174:0/949646328 >> v2:172.21.15.174:6800/1252702393 conn(0x7facfc06d9f0 0x7facfc06feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:04.391 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.388+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=767}) v3 -- 0x7facfc078fc0 con 0x7fad10100380 2024-02-10T07:02:04.391 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.388+0000 7fad07fff640 1 --2- 172.21.15.174:0/949646328 >> v2:172.21.15.174:6800/1252702393 conn(0x7facfc06d9f0 0x7facfc06feb0 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7fad1006a480 tx=0x7facf800b410 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:04.393 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.393+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad0005acf0 con 0x7fad10100380 2024-02-10T07:02:04.571 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:04.570+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fad10100d30 con 0x7fad10100380 2024-02-10T07:02:04.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:04 smithi184 ceph-mon[33513]: pgmap v1215: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:04.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:02:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:04 smithi184 ceph-mon[33513]: osdmap e765: 8 total, 8 up, 8 in 2024-02-10T07:02:04.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:04 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:02:04.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[29878]: pgmap v1215: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[29878]: osdmap e765: 8 total, 8 up, 8 in 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[39523]: pgmap v1215: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[39523]: osdmap e765: 8 total, 8 up, 8 in 2024-02-10T07:02:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:04 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:02:05.012 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:05.011+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 7 ==== osd_map(767..767 src has 1..767) v4 ==== 628+0+0 (secure 0 0 0) 0x7fad0005efc0 con 0x7fad10100380 2024-02-10T07:02:05.013 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:05.011+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=768}) v3 -- 0x7facfc079af0 con 0x7fad10100380 2024-02-10T07:02:05.017 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:05.016+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v767) v1 ==== 221+0+0 (secure 0 0 0) 0x7fad0005a880 con 0x7fad10100380 2024-02-10T07:02:05.124 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:05.123+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fad10069f20 con 0x7fad10100380 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: osdmap e766: 8 total, 8 up, 8 in 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/949646328' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: pool '37577e97-8613-4d15-a71c-97ee4e569923' no longer out of quota; removing NO_QUOTA flag 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: osdmap e767: 8 total, 8 up, 8 in 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/949646328' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:05 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: osdmap e766: 8 total, 8 up, 8 in 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/949646328' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: pool '37577e97-8613-4d15-a71c-97ee4e569923' no longer out of quota; removing NO_QUOTA flag 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: osdmap e767: 8 total, 8 up, 8 in 2024-02-10T07:02:05.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/949646328' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/762256297' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: osdmap e766: 8 total, 8 up, 8 in 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/949646328' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: pool '37577e97-8613-4d15-a71c-97ee4e569923' no longer out of quota; removing NO_QUOTA flag 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: osdmap e767: 8 total, 8 up, 8 in 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/949646328' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:05.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:05 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:02:06.031 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.031+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 9 ==== osd_map(768..768 src has 1..768) v4 ==== 628+0+0 (secure 0 0 0) 0x7fad0005a880 con 0x7fad10100380 2024-02-10T07:02:06.036 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.035+0000 7fad077fe640 1 -- 172.21.15.174:0/949646328 <== mon.2 v2:172.21.15.174:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v768) v1 ==== 221+0+0 (secure 0 0 0) 0x7fad0005aa60 con 0x7fad10100380 2024-02-10T07:02:06.036 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_objects = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 >> v2:172.21.15.174:6800/1252702393 conn(0x7facfc06d9f0 msgr2=0x7facfc06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 --2- 172.21.15.174:0/949646328 >> v2:172.21.15.174:6800/1252702393 conn(0x7facfc06d9f0 0x7facfc06feb0 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7fad1006a480 tx=0x7facf800b410 comp rx=0 tx=0).stop 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 msgr2=0x7fad10197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 0x7fad10197d70 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e9f0 tx=0x7fad0000eeb0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 shutdown_connections 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 --2- 172.21.15.174:0/949646328 >> v2:172.21.15.174:6800/1252702393 conn(0x7facfc06d9f0 0x7facfc06feb0 unknown :-1 s=CLOSED pgs=3457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad10198840 0x7fad101a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.040 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad10105450 0x7fad101982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.041 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 --2- 172.21.15.174:0/949646328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fad10100380 0x7fad10197d70 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.041 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 >> 172.21.15.174:0/949646328 conn(0x7fad10077310 msgr2=0x7fad100fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:06.041 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 shutdown_connections 2024-02-10T07:02:06.041 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.039+0000 7fad14d7a640 1 -- 172.21.15.174:0/949646328 wait complete. 2024-02-10T07:02:06.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ wait 151000 2024-02-10T07:02:06.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ '[' 0 -ne 0 ']' 2024-02-10T07:02:06.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ true 2024-02-10T07:02:06.051 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put three /etc/passwd 2024-02-10T07:02:06.089 INFO:tasks.workunit.client.0.smithi174.stderr:++ uuidgen 2024-02-10T07:02:06.089 INFO:tasks.workunit.client.0.smithi174.stderr:+ pp=fd43c8b8-70b8-4fd9-961b-3c27c45d6680 2024-02-10T07:02:06.089 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool create fd43c8b8-70b8-4fd9-961b-3c27c45d6680 12 2024-02-10T07:02:06.175 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 -- 172.21.15.174:0/2848782404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 msgr2=0x7f2d10100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 --2- 172.21.15.174:0/2848782404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10100000 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009930 tx=0x7f2d0402bcd0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 -- 172.21.15.174:0/2848782404 shutdown_connections 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 --2- 172.21.15.174:0/2848782404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d10100540 0x7f2d1010c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 --2- 172.21.15.174:0/2848782404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10100000 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 --2- 172.21.15.174:0/2848782404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d10105970 0x7f2d10105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 -- 172.21.15.174:0/2848782404 >> 172.21.15.174:0/2848782404 conn(0x7f2d10075410 msgr2=0x7f2d10075820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 -- 172.21.15.174:0/2848782404 shutdown_connections 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 -- 172.21.15.174:0/2848782404 wait complete. 2024-02-10T07:02:06.176 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 Processor -- start 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 -- start start 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.173+0000 7f2d15be9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d10100540 0x7f2d101952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d10105970 0x7f2d10199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10199c70 con 0x7f2d10100540 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10199de0 con 0x7f2d10105970 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d10199fa0 con 0x7f2d100ffba0 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d10100540 0x7f2d101952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10194db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:44316/0 (socket says 172.21.15.174:44316) 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 -- 172.21.15.174:0/3799277526 learned_addr learned my addr 172.21.15.174:0/3799277526 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0ffff640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d10105970 0x7f2d10199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 -- 172.21.15.174:0/3799277526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d10105970 msgr2=0x7f2d10199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d10105970 0x7f2d10199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 -- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d10100540 msgr2=0x7f2d101952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:06.177 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d10100540 0x7f2d101952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 -- 172.21.15.174:0/3799277526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0800a950 con 0x7f2d100ffba0 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0f7fe640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10194db0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000d770 tx=0x7f2d0000db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00005220 con 0x7f2d100ffba0 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f2d00009cd0 con 0x7f2d100ffba0 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d04009560 con 0x7f2d100ffba0 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004d00 con 0x7f2d100ffba0 2024-02-10T07:02:06.178 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.174+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d101a1920 con 0x7f2d100ffba0 2024-02-10T07:02:06.179 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.175+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d10068560 con 0x7f2d100ffba0 2024-02-10T07:02:06.179 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.176+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f2d00009820 con 0x7f2d100ffba0 2024-02-10T07:02:06.179 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.176+0000 7f2d0cff9640 1 --2- 172.21.15.174:0/3799277526 >> v2:172.21.15.174:6800/1252702393 conn(0x7f2cf806d9f0 0x7f2cf806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:06.179 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.176+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(768..768 src has 1..768) v4 ==== 7843+0+0 (secure 0 0 0) 0x7f2d0008d400 con 0x7f2d100ffba0 2024-02-10T07:02:06.179 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.176+0000 7f2d0effd640 1 --2- 172.21.15.174:0/3799277526 >> v2:172.21.15.174:6800/1252702393 conn(0x7f2cf806d9f0 0x7f2cf806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:06.179 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.176+0000 7f2d0effd640 1 --2- 172.21.15.174:0/3799277526 >> v2:172.21.15.174:6800/1252702393 conn(0x7f2cf806d9f0 0x7f2cf806feb0 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005ab0 tx=0x7f2d04005fd0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:06.181 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.181+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d0005a2a0 con 0x7f2d100ffba0 2024-02-10T07:02:06.357 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:06.356+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12} v 0) v1 -- 0x7f2d10107d80 con 0x7f2d100ffba0 2024-02-10T07:02:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:06 smithi184 ceph-mon[33513]: pgmap v1219: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:06 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:02:06.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:06 smithi184 ceph-mon[33513]: osdmap e768: 8 total, 8 up, 8 in 2024-02-10T07:02:06.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:06 smithi174 ceph-mon[29878]: pgmap v1219: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:06 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:02:06.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:06 smithi174 ceph-mon[29878]: osdmap e768: 8 total, 8 up, 8 in 2024-02-10T07:02:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:06 smithi174 ceph-mon[39523]: pgmap v1219: 176 pgs: 176 active+clean; 720 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:06 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:02:06.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:06 smithi174 ceph-mon[39523]: osdmap e768: 8 total, 8 up, 8 in 2024-02-10T07:02:07.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:02:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:02:07.297 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.297+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]=0 pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' created v769) v1 ==== 176+0+0 (secure 0 0 0) 0x7f2d00059e30 con 0x7f2d100ffba0 2024-02-10T07:02:07.406 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.406+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12} v 0) v1 -- 0x7f2d1010ea30 con 0x7f2d100ffba0 2024-02-10T07:02:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.407+0000 7f2d0cff9640 1 -- 172.21.15.174:0/3799277526 <== mon.2 v2:172.21.15.174:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]=0 pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' already exists v769) v1 ==== 183+0+0 (secure 0 0 0) 0x7f2d00020070 con 0x7f2d100ffba0 2024-02-10T07:02:07.407 INFO:tasks.workunit.client.0.smithi174.stderr:pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' already exists 2024-02-10T07:02:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 >> v2:172.21.15.174:6800/1252702393 conn(0x7f2cf806d9f0 msgr2=0x7f2cf806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:07.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 --2- 172.21.15.174:0/3799277526 >> v2:172.21.15.174:6800/1252702393 conn(0x7f2cf806d9f0 0x7f2cf806feb0 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005ab0 tx=0x7f2d04005fd0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 msgr2=0x7f2d10194db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10194db0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000d770 tx=0x7f2d0000db70 comp rx=0 tx=0).stop 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 shutdown_connections 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 --2- 172.21.15.174:0/3799277526 >> v2:172.21.15.174:6800/1252702393 conn(0x7f2cf806d9f0 0x7f2cf806feb0 unknown :-1 s=CLOSED pgs=3459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d10105970 0x7f2d10199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d10100540 0x7f2d101952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 --2- 172.21.15.174:0/3799277526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d100ffba0 0x7f2d10194db0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 >> 172.21.15.174:0/3799277526 conn(0x7f2d10075410 msgr2=0x7f2d100fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 shutdown_connections 2024-02-10T07:02:07.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.409+0000 7f2d15be9640 1 -- 172.21.15.174:0/3799277526 wait complete. 2024-02-10T07:02:07.418 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool application enable fd43c8b8-70b8-4fd9-961b-3c27c45d6680 rados 2024-02-10T07:02:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/314009674 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb78100360 msgr2=0x7fbb78100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/314009674 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb78100360 0x7fbb78100740 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009ca0 tx=0x7fbb6802c0f0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/314009674 shutdown_connections 2024-02-10T07:02:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/314009674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb78105430 0x7fbb7810d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.503 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/314009674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100d10 0x7fbb78104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/314009674 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb78100360 0x7fbb78100740 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/314009674 >> 172.21.15.174:0/314009674 conn(0x7fbb78077250 msgr2=0x7fbb78077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/314009674 shutdown_connections 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.500+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/314009674 wait complete. 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 Processor -- start 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 -- start start 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 0x7fbb78196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb78105430 0x7fbb78197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb7819b880 0x7fbb7819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7819c270 con 0x7fbb78100360 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7819c3e0 con 0x7fbb78105430 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7ed3a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7819c550 con 0x7fbb7819b880 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7cad6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 0x7fbb78196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:07.504 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7cad6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 0x7fbb78196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50150/0 (socket says 172.21.15.174:50150) 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7cad6640 1 -- 172.21.15.174:0/2827490439 learned_addr learned my addr 172.21.15.174:0/2827490439 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb7d2d7640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb7819b880 0x7fbb7819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.501+0000 7fbb77fff640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb78105430 0x7fbb78197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7cad6640 1 -- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb7819b880 msgr2=0x7fbb7819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7cad6640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb7819b880 0x7fbb7819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7cad6640 1 -- 172.21.15.174:0/2827490439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb78105430 msgr2=0x7fbb78197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7cad6640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb78105430 0x7fbb78197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7cad6640 1 -- 172.21.15.174:0/2827490439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60009490 con 0x7fbb78100360 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7d2d7640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb7819b880 0x7fbb7819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7cad6640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 0x7fbb78196fc0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fbb68005e30 tx=0x7fbb68032960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68034c70 con 0x7fbb78100360 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb68009900 con 0x7fbb78100360 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fbb68003e20 con 0x7fbb78100360 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680047f0 con 0x7fbb78100360 2024-02-10T07:02:07.505 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.502+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb781a3dc0 con 0x7fbb78100360 2024-02-10T07:02:07.506 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.503+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb78068110 con 0x7fbb78100360 2024-02-10T07:02:07.506 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.504+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fbb6803b040 con 0x7fbb78100360 2024-02-10T07:02:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.504+0000 7fbb75ffb640 1 --2- 172.21.15.174:0/2827490439 >> v2:172.21.15.174:6800/1252702393 conn(0x7fbb6406d9f0 0x7fbb6406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.504+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(769..769 src has 1..769) v4 ==== 8218+0+0 (secure 0 0 0) 0x7fbb680b04a0 con 0x7fbb78100360 2024-02-10T07:02:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.504+0000 7fbb77fff640 1 --2- 172.21.15.174:0/2827490439 >> v2:172.21.15.174:6800/1252702393 conn(0x7fbb6406d9f0 0x7fbb6406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:07.507 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.504+0000 7fbb77fff640 1 --2- 172.21.15.174:0/2827490439 >> v2:172.21.15.174:6800/1252702393 conn(0x7fbb6406d9f0 0x7fbb6406feb0 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ffca0 tx=0x7fbb60005f90 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:07.509 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.509+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb6807d180 con 0x7fbb78100360 2024-02-10T07:02:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:07 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3799277526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:07 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:07.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:07 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:07.684 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:07.684+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"} v 0) v1 -- 0x7fbb78100d10 con 0x7fbb78100360 2024-02-10T07:02:07.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3799277526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3799277526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:07.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:08.238 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:02:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:02:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:02:08.295 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:08.295+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]=0 enabled application 'rados' on pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' v770) v1 ==== 213+0+0 (secure 0 0 0) 0x7fbb6807cd10 con 0x7fbb78100360 2024-02-10T07:02:08.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:08.412+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"} v 0) v1 -- 0x7fbb78110aa0 con 0x7fbb78100360 2024-02-10T07:02:08.625 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:08 smithi184 ceph-mon[33513]: pgmap v1221: 176 pgs: 176 active+clean; 731 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T07:02:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]': finished 2024-02-10T07:02:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:08 smithi184 ceph-mon[33513]: osdmap e769: 8 total, 8 up, 8 in 2024-02-10T07:02:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3799277526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:08 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:08.626 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:08 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]: dispatch 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[29878]: pgmap v1221: 176 pgs: 176 active+clean; 731 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]': finished 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[29878]: osdmap e769: 8 total, 8 up, 8 in 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3799277526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]: dispatch 2024-02-10T07:02:08.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[39523]: pgmap v1221: 176 pgs: 176 active+clean; 731 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T07:02:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]': finished 2024-02-10T07:02:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[39523]: osdmap e769: 8 total, 8 up, 8 in 2024-02-10T07:02:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3799277526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pg_num": 12}]: dispatch 2024-02-10T07:02:08.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:08 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]: dispatch 2024-02-10T07:02:09.297 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.296+0000 7fbb75ffb640 1 -- 172.21.15.174:0/2827490439 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]=0 enabled application 'rados' on pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' v771) v1 ==== 213+0+0 (secure 0 0 0) 0x7fbb680410d0 con 0x7fbb78100360 2024-02-10T07:02:09.297 INFO:tasks.workunit.client.0.smithi174.stderr:enabled application 'rados' on pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' 2024-02-10T07:02:09.300 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 >> v2:172.21.15.174:6800/1252702393 conn(0x7fbb6406d9f0 msgr2=0x7fbb6406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:09.300 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/2827490439 >> v2:172.21.15.174:6800/1252702393 conn(0x7fbb6406d9f0 0x7fbb6406feb0 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ffca0 tx=0x7fbb60005f90 comp rx=0 tx=0).stop 2024-02-10T07:02:09.300 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 msgr2=0x7fbb78196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:09.300 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 0x7fbb78196fc0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fbb68005e30 tx=0x7fbb68032960 comp rx=0 tx=0).stop 2024-02-10T07:02:09.300 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 shutdown_connections 2024-02-10T07:02:09.300 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/2827490439 >> v2:172.21.15.174:6800/1252702393 conn(0x7fbb6406d9f0 0x7fbb6406feb0 secure :-1 s=CLOSED pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7fbb780ffca0 tx=0x7fbb60005f90 comp rx=0 tx=0).stop 2024-02-10T07:02:09.301 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb7819b880 0x7fbb7819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.301 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb78105430 0x7fbb78197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.301 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.299+0000 7fbb7ed3a640 1 --2- 172.21.15.174:0/2827490439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb78100360 0x7fbb78196fc0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.301 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.300+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 >> 172.21.15.174:0/2827490439 conn(0x7fbb78077250 msgr2=0x7fbb780fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:09.301 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.300+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 shutdown_connections 2024-02-10T07:02:09.301 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.300+0000 7fbb7ed3a640 1 -- 172.21.15.174:0/2827490439 wait complete. 2024-02-10T07:02:09.319 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota fd43c8b8-70b8-4fd9-961b-3c27c45d6680 max_objects 10 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- 172.21.15.174:0/2308448166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08100d30 msgr2=0x7faf08104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- 172.21.15.174:0/2308448166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08100d30 0x7faf08104d70 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7faef8009930 tx=0x7faef802bc30 comp rx=0 tx=0).stop 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- 172.21.15.174:0/2308448166 shutdown_connections 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- 172.21.15.174:0/2308448166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf08105450 0x7faf0810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- 172.21.15.174:0/2308448166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08100d30 0x7faf08104d70 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- 172.21.15.174:0/2308448166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 0x7faf08100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.409 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- 172.21.15.174:0/2308448166 >> 172.21.15.174:0/2308448166 conn(0x7faf08077310 msgr2=0x7faf08077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- 172.21.15.174:0/2308448166 shutdown_connections 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- 172.21.15.174:0/2308448166 wait complete. 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 Processor -- start 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- start start 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 0x7faf08197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 0x7faf08197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 0x7faf0819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.404+0000 7faf1018c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0819c300 con 0x7faf08105450 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf1018c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0819c470 con 0x7faf08100380 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf1018c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0819c5e0 con 0x7faf0819b910 2024-02-10T07:02:09.410 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 0x7faf08197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0e729640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 0x7faf0819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 0x7faf08197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50174/0 (socket says 172.21.15.174:50174) 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0e729640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 0x7faf0819bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:44360/0 (socket says 172.21.15.174:44360) 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 -- 172.21.15.174:0/3570168032 learned_addr learned my addr 172.21.15.174:0/3570168032 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0df28640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 0x7faf08197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 -- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 msgr2=0x7faf0819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 0x7faf0819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 -- 172.21.15.174:0/3570168032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 msgr2=0x7faf08197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 0x7faf08197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:09.411 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 -- 172.21.15.174:0/3570168032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf00009840 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0e729640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 0x7faf0819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0df28640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 0x7faf08197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf0d727640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 0x7faf08197590 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7faef8009230 tx=0x7faef8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8004440 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7faef8003aa0 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8009560 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef80053f0 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.405+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf081a3ee0 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.406+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf08068560 con 0x7faf08105450 2024-02-10T07:02:09.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.407+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7faef8005550 con 0x7faf08105450 2024-02-10T07:02:09.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.407+0000 7faefeffd640 1 --2- 172.21.15.174:0/3570168032 >> v2:172.21.15.174:6800/1252702393 conn(0x7faef406d9f0 0x7faef406feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:02:09.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.407+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(771..771 src has 1..771) v4 ==== 8231+0+0 (secure 0 0 0) 0x7faef80afc10 con 0x7faf08105450 2024-02-10T07:02:09.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.407+0000 7faf0df28640 1 --2- 172.21.15.174:0/3570168032 >> v2:172.21.15.174:6800/1252702393 conn(0x7faef406d9f0 0x7faef406feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:02:09.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.408+0000 7faf0df28640 1 --2- 172.21.15.174:0/3570168032 >> v2:172.21.15.174:6800/1252702393 conn(0x7faef406d9f0 0x7faef406feb0 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 crypto rx=0x7faf04009b40 tx=0x7faf04005e50 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:02:09.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.412+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faef807c8b0 con 0x7faf08105450 2024-02-10T07:02:09.588 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:09.587+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7faf08100d30 con 0x7faf08105450 2024-02-10T07:02:09.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]': finished 2024-02-10T07:02:09.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:09 smithi184 ceph-mon[33513]: osdmap e770: 8 total, 8 up, 8 in 2024-02-10T07:02:09.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]: dispatch 2024-02-10T07:02:09.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:09 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]': finished 2024-02-10T07:02:09.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:09 smithi184 ceph-mon[33513]: osdmap e771: 8 total, 8 up, 8 in 2024-02-10T07:02:09.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]': finished 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[29878]: osdmap e770: 8 total, 8 up, 8 in 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]: dispatch 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]': finished 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[29878]: osdmap e771: 8 total, 8 up, 8 in 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]': finished 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[39523]: osdmap e770: 8 total, 8 up, 8 in 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]: dispatch 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2827490439' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "app": "rados"}]': finished 2024-02-10T07:02:09.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:09 smithi174 ceph-mon[39523]: osdmap e771: 8 total, 8 up, 8 in 2024-02-10T07:02:10.321 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:10.320+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool fd43c8b8-70b8-4fd9-961b-3c27c45d6680 v772) v1 ==== 223+0+0 (secure 0 0 0) 0x7faef807c440 con 0x7faf08105450 2024-02-10T07:02:10.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:10.431+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7faf0810f900 con 0x7faf08105450 2024-02-10T07:02:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:10 smithi184 ceph-mon[33513]: pgmap v1224: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-10T07:02:10.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:10 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:02:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:10 smithi174 ceph-mon[29878]: pgmap v1224: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-10T07:02:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:10 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:02:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:10 smithi174 ceph-mon[39523]: pgmap v1224: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 817 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-10T07:02:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:10 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:02:11.326 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.325+0000 7faefeffd640 1 -- 172.21.15.174:0/3570168032 <== mon.0 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool fd43c8b8-70b8-4fd9-961b-3c27c45d6680 v773) v1 ==== 223+0+0 (secure 0 0 0) 0x7faef8043030 con 0x7faf08105450 2024-02-10T07:02:11.326 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_objects = 10 for pool fd43c8b8-70b8-4fd9-961b-3c27c45d6680 2024-02-10T07:02:11.329 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 >> v2:172.21.15.174:6800/1252702393 conn(0x7faef406d9f0 msgr2=0x7faef406feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:11.329 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 --2- 172.21.15.174:0/3570168032 >> v2:172.21.15.174:6800/1252702393 conn(0x7faef406d9f0 0x7faef406feb0 secure :-1 s=READY pgs=3461 cs=0 l=1 rev1=1 crypto rx=0x7faf04009b40 tx=0x7faf04005e50 comp rx=0 tx=0).stop 2024-02-10T07:02:11.329 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 msgr2=0x7faf08197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:02:11.329 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 0x7faf08197590 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7faef8009230 tx=0x7faef8004040 comp rx=0 tx=0).stop 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 shutdown_connections 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 --2- 172.21.15.174:0/3570168032 >> v2:172.21.15.174:6800/1252702393 conn(0x7faef406d9f0 0x7faef406feb0 unknown :-1 s=CLOSED pgs=3461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf0819b910 0x7faf0819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf08105450 0x7faf08197590 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 --2- 172.21.15.174:0/3570168032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf08100380 0x7faf08197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 >> 172.21.15.174:0/3570168032 conn(0x7faf08077310 msgr2=0x7faf08104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.329+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 shutdown_connections 2024-02-10T07:02:11.330 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:02:11.330+0000 7faf1018c640 1 -- 172.21.15.174:0/3570168032 wait complete. 2024-02-10T07:02:11.342 INFO:tasks.workunit.client.0.smithi174.stderr:+ sleep 30 2024-02-10T07:02:11.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:02:11.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:11 smithi184 ceph-mon[33513]: osdmap e772: 8 total, 8 up, 8 in 2024-02-10T07:02:11.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:11 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:02:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:02:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:11 smithi174 ceph-mon[29878]: osdmap e772: 8 total, 8 up, 8 in 2024-02-10T07:02:11.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:11 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:02:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:02:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:11 smithi174 ceph-mon[39523]: osdmap e772: 8 total, 8 up, 8 in 2024-02-10T07:02:11.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:11 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T07:02:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:12 smithi184 ceph-mon[33513]: pgmap v1227: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T07:02:12.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:02:12.624 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:12 smithi184 ceph-mon[33513]: osdmap e773: 8 total, 8 up, 8 in 2024-02-10T07:02:12.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:12 smithi174 ceph-mon[29878]: pgmap v1227: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T07:02:12.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:02:12.741 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:12 smithi174 ceph-mon[29878]: osdmap e773: 8 total, 8 up, 8 in 2024-02-10T07:02:12.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:12 smithi174 ceph-mon[39523]: pgmap v1227: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T07:02:12.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3570168032' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "field": "max_objects", "val": "10"}]': finished 2024-02-10T07:02:12.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:12 smithi174 ceph-mon[39523]: osdmap e773: 8 total, 8 up, 8 in 2024-02-10T07:02:14.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:14 smithi184 ceph-mon[33513]: pgmap v1229: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T07:02:14.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:14 smithi174 ceph-mon[29878]: pgmap v1229: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T07:02:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:14 smithi174 ceph-mon[39523]: pgmap v1229: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T07:02:16.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:16 smithi184 ceph-mon[33513]: pgmap v1230: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T07:02:16.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:16 smithi174 ceph-mon[29878]: pgmap v1230: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T07:02:16.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:16 smithi174 ceph-mon[39523]: pgmap v1230: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T07:02:17.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:17.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:02:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:02:17.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:17.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:18.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:18 smithi174 ceph-mon[29878]: pgmap v1231: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:02:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:02:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:02:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:02:18.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:18 smithi174 ceph-mon[39523]: pgmap v1231: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:02:18.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:18 smithi184 ceph-mon[33513]: pgmap v1231: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:02:20.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:20 smithi184 ceph-mon[33513]: pgmap v1232: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-10T07:02:20.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:20 smithi174 ceph-mon[29878]: pgmap v1232: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-10T07:02:20.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:20 smithi174 ceph-mon[39523]: pgmap v1232: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-10T07:02:22.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:22 smithi184 ceph-mon[33513]: pgmap v1233: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:02:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:22 smithi174 ceph-mon[29878]: pgmap v1233: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:02:22.740 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:22 smithi174 ceph-mon[39523]: pgmap v1233: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:02:22.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:24.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:24 smithi184 ceph-mon[33513]: pgmap v1234: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 861 B/s rd, 0 op/s 2024-02-10T07:02:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:24 smithi174 ceph-mon[29878]: pgmap v1234: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 861 B/s rd, 0 op/s 2024-02-10T07:02:24.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:24 smithi174 ceph-mon[39523]: pgmap v1234: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 861 B/s rd, 0 op/s 2024-02-10T07:02:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:26 smithi184 ceph-mon[33513]: pgmap v1235: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:26.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:26 smithi174 ceph-mon[29878]: pgmap v1235: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:26 smithi174 ceph-mon[39523]: pgmap v1235: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:27.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:02:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:02:27.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:28.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:28 smithi174 ceph-mon[29878]: pgmap v1236: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:02:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:02:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:02:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:28 smithi174 ceph-mon[39523]: pgmap v1236: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:28 smithi184 ceph-mon[33513]: pgmap v1236: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:30 smithi184 ceph-mon[33513]: pgmap v1237: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:30.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:30 smithi174 ceph-mon[29878]: pgmap v1237: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:30.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:30 smithi174 ceph-mon[39523]: pgmap v1237: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:32 smithi184 ceph-mon[33513]: pgmap v1238: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:32.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:32 smithi174 ceph-mon[29878]: pgmap v1238: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:32.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:32 smithi174 ceph-mon[39523]: pgmap v1238: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:34.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:34 smithi184 ceph-mon[33513]: pgmap v1239: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:34.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:34 smithi174 ceph-mon[29878]: pgmap v1239: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:34 smithi174 ceph-mon[39523]: pgmap v1239: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:36.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:36 smithi184 ceph-mon[33513]: pgmap v1240: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:36 smithi174 ceph-mon[29878]: pgmap v1240: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:36 smithi174 ceph-mon[39523]: pgmap v1240: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:37.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:37.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:02:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:02:37.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:38 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:38 smithi174 ceph-mon[29878]: pgmap v1241: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:02:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:02:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:02:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:38 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:38.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:38 smithi174 ceph-mon[39523]: pgmap v1241: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:38 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:38 smithi184 ceph-mon[33513]: pgmap v1241: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:40.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:40 smithi184 ceph-mon[33513]: pgmap v1242: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:40.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:40 smithi174 ceph-mon[29878]: pgmap v1242: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:40 smithi174 ceph-mon[39523]: pgmap v1242: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:41.344 INFO:tasks.workunit.client.0.smithi174.stderr:++ seq 1 10 2024-02-10T07:02:41.345 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.345 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj1 /etc/passwd 2024-02-10T07:02:41.386 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.386 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj2 /etc/passwd 2024-02-10T07:02:41.423 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.423 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj3 /etc/passwd 2024-02-10T07:02:41.459 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.459 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj4 /etc/passwd 2024-02-10T07:02:41.494 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.495 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj5 /etc/passwd 2024-02-10T07:02:41.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.529 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj6 /etc/passwd 2024-02-10T07:02:41.563 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.563 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj7 /etc/passwd 2024-02-10T07:02:41.598 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.598 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj8 /etc/passwd 2024-02-10T07:02:41.632 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.632 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj9 /etc/passwd 2024-02-10T07:02:41.666 INFO:tasks.workunit.client.0.smithi174.stderr:+ for f in `seq 1 10` 2024-02-10T07:02:41.666 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p fd43c8b8-70b8-4fd9-961b-3c27c45d6680 put obj10 /etc/passwd 2024-02-10T07:02:41.701 INFO:tasks.workunit.client.0.smithi174.stderr:+ sleep 30 2024-02-10T07:02:42.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:42 smithi184 ceph-mon[33513]: pgmap v1243: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:42.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:42 smithi174 ceph-mon[29878]: pgmap v1243: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:42 smithi174 ceph-mon[39523]: pgmap v1243: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:02:44.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:44 smithi184 ceph-mon[33513]: pgmap v1244: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:44.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:44 smithi174 ceph-mon[29878]: pgmap v1244: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:44 smithi174 ceph-mon[39523]: pgmap v1244: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:46.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:46 smithi184 ceph-mon[33513]: pgmap v1245: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:46.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:46 smithi174 ceph-mon[29878]: pgmap v1245: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:46 smithi174 ceph-mon[39523]: pgmap v1245: 188 pgs: 188 active+clean; 742 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:02:47.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:02:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:02:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:47 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:02:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:47 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:02:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:47 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:02:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:47 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:02:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:02:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:02:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:47.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:02:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:02:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:02:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:47.490 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:48.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:48 smithi174 ceph-mon[29878]: pgmap v1246: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T07:02:48.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:02:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:02:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:02:48.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:48 smithi174 ceph-mon[39523]: pgmap v1246: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T07:02:48.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:48 smithi184 ceph-mon[33513]: pgmap v1246: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T07:02:50.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:50 smithi184 ceph-mon[33513]: pgmap v1247: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T07:02:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:50 smithi184 ceph-mon[33513]: pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' is full (reached quota's max_objects: 10) 2024-02-10T07:02:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:50 smithi184 ceph-mon[33513]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:02:50.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:50 smithi184 ceph-mon[33513]: osdmap e774: 8 total, 8 up, 8 in 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[29878]: pgmap v1247: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[29878]: pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' is full (reached quota's max_objects: 10) 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[29878]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[29878]: osdmap e774: 8 total, 8 up, 8 in 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[39523]: pgmap v1247: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[39523]: pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' is full (reached quota's max_objects: 10) 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[39523]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:02:50.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:50 smithi174 ceph-mon[39523]: osdmap e774: 8 total, 8 up, 8 in 2024-02-10T07:02:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:52 smithi184 ceph-mon[33513]: pgmap v1249: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:52.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:52 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:52.738 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:52 smithi174 ceph-mon[39523]: pgmap v1249: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:52.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:52 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:52 smithi174 ceph-mon[29878]: pgmap v1249: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:02:52.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:52 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:02:54.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:54 smithi184 ceph-mon[33513]: pgmap v1250: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:54.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:54 smithi174 ceph-mon[29878]: pgmap v1250: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:54.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:54 smithi174 ceph-mon[39523]: pgmap v1250: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:56.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:56 smithi184 ceph-mon[33513]: pgmap v1251: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:56.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:56 smithi174 ceph-mon[29878]: pgmap v1251: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:56.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:56 smithi174 ceph-mon[39523]: pgmap v1251: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T07:02:57.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:57 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:57.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:02:56 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:02:57.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:57 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:57.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:57 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:02:58.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:02:58 smithi174 ceph-mon[29878]: pgmap v1252: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:02:58.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:02:57 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:02:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:02:58.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:02:58 smithi174 ceph-mon[39523]: pgmap v1252: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:02:58.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:02:58 smithi184 ceph-mon[33513]: pgmap v1252: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:00.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:00 smithi184 ceph-mon[33513]: pgmap v1253: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:00.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:00 smithi174 ceph-mon[29878]: pgmap v1253: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:00.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:00 smithi174 ceph-mon[39523]: pgmap v1253: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:02.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:02 smithi184 ceph-mon[33513]: pgmap v1254: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T07:03:02.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:02 smithi174 ceph-mon[29878]: pgmap v1254: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T07:03:02.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:02 smithi174 ceph-mon[39523]: pgmap v1254: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T07:03:04.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:04 smithi184 ceph-mon[33513]: pgmap v1255: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:03:04.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:04 smithi174 ceph-mon[29878]: pgmap v1255: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:03:04.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:04 smithi174 ceph-mon[39523]: pgmap v1255: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:03:06.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:06 smithi184 ceph-mon[33513]: pgmap v1256: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:06.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:06 smithi174 ceph-mon[29878]: pgmap v1256: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:06.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:06 smithi174 ceph-mon[39523]: pgmap v1256: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:07.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:07 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:07.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:03:06 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:03:07.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:07 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:07.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:07 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:08 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:08.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:08 smithi174 ceph-mon[29878]: pgmap v1257: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:08.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:07 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:03:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:03:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:08 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:08.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:08 smithi174 ceph-mon[39523]: pgmap v1257: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:08.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:08 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:08.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:08 smithi184 ceph-mon[33513]: pgmap v1257: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:10.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:10 smithi184 ceph-mon[33513]: pgmap v1258: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:03:10.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:10 smithi174 ceph-mon[29878]: pgmap v1258: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:03:10.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:10 smithi174 ceph-mon[39523]: pgmap v1258: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:03:11.703 INFO:tasks.workunit.client.0.smithi174.stderr:+ rados -p 37577e97-8613-4d15-a71c-97ee4e569923 put threemore /etc/passwd 2024-02-10T07:03:11.743 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_bytes 0 2024-02-10T07:03:11.834 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 -- 172.21.15.174:0/311440871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c100d10 msgr2=0x7fa72c104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:11.834 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 --2- 172.21.15.174:0/311440871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c100d10 0x7fa72c104d50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fa718009930 tx=0x7fa71802bc30 comp rx=0 tx=0).stop 2024-02-10T07:03:11.834 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 -- 172.21.15.174:0/311440871 shutdown_connections 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 --2- 172.21.15.174:0/311440871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa72c105430 0x7fa72c10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 --2- 172.21.15.174:0/311440871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c100d10 0x7fa72c104d50 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 --2- 172.21.15.174:0/311440871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c100360 0x7fa72c100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 -- 172.21.15.174:0/311440871 >> 172.21.15.174:0/311440871 conn(0x7fa72c077250 msgr2=0x7fa72c077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 -- 172.21.15.174:0/311440871 shutdown_connections 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 -- 172.21.15.174:0/311440871 wait complete. 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 Processor -- start 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.830+0000 7fa7330ce640 1 -- start start 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa72c100360 0x7fa72c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 0x7fa72c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c1988c0 0x7fa72c1a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1a3530 con 0x7fa72c1988c0 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1a36a0 con 0x7fa72c100360 2024-02-10T07:03:11.835 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1a3860 con 0x7fa72c105430 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 0x7fa72c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa73166b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c1988c0 0x7fa72c1a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 0x7fa72c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:34694/0 (socket says 172.21.15.174:34694) 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 -- 172.21.15.174:0/1881644244 learned_addr learned my addr 172.21.15.174:0/1881644244 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa730e6a640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa72c100360 0x7fa72c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 -- 172.21.15.174:0/1881644244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa72c100360 msgr2=0x7fa72c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa72c100360 0x7fa72c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 -- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c1988c0 msgr2=0x7fa72c1a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c1988c0 0x7fa72c1a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:11.836 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720009490 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa73166b640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c1988c0 0x7fa72c1a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa72bfff640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 0x7fa72c198380 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fa718009230 tx=0x7fa718004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718004440 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa718009560 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fa718003aa0 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.831+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718005460 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.832+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c1a3eb0 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.833+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c068110 con 0x7fa72c105430 2024-02-10T07:03:11.837 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.833+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fa7180035f0 con 0x7fa72c105430 2024-02-10T07:03:11.838 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.834+0000 7fa729ffb640 1 --2- 172.21.15.174:0/1881644244 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa71c06da20 0x7fa71c06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:11.838 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.834+0000 7fa730e6a640 1 --2- 172.21.15.174:0/1881644244 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa71c06da20 0x7fa71c06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:11.838 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.834+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(774..774 src has 1..774) v4 ==== 8231+0+0 (secure 0 0 0) 0x7fa7180afbe0 con 0x7fa72c105430 2024-02-10T07:03:11.838 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.834+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7fa71c079810 con 0x7fa72c105430 2024-02-10T07:03:11.838 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.834+0000 7fa730e6a640 1 --2- 172.21.15.174:0/1881644244 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa71c06da20 0x7fa71c06fee0 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d60 tx=0x7fa720005f90 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:11.841 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:11.838+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa71807c8b0 con 0x7fa72c105430 2024-02-10T07:03:12.014 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:12.013+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fa72c100d10 con 0x7fa72c105430 2024-02-10T07:03:12.289 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:12.287+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 7 ==== osd_map(775..775 src has 1..775) v4 ==== 628+0+0 (secure 0 0 0) 0x7fa718003c10 con 0x7fa72c105430 2024-02-10T07:03:12.289 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:12.287+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=776}) v3 -- 0x7fa71c07a380 con 0x7fa72c105430 2024-02-10T07:03:12.292 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:12.291+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v775) v1 ==== 217+0+0 (secure 0 0 0) 0x7fa718043070 con 0x7fa72c105430 2024-02-10T07:03:12.405 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:12.403+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fa72c100ef0 con 0x7fa72c105430 2024-02-10T07:03:12.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:12 smithi184 ceph-mon[33513]: pgmap v1259: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:12 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1881644244' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:12.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:12 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:12.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:12 smithi174 ceph-mon[29878]: pgmap v1259: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:12 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1881644244' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:12.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:12 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:12 smithi174 ceph-mon[39523]: pgmap v1259: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:12 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1881644244' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:12.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:12 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:13.302 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.301+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 9 ==== osd_map(776..776 src has 1..776) v4 ==== 628+0+0 (secure 0 0 0) 0x7fa718080b80 con 0x7fa72c105430 2024-02-10T07:03:13.303 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.301+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=777}) v3 -- 0x7fa71c07a7f0 con 0x7fa72c105430 2024-02-10T07:03:13.307 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.305+0000 7fa729ffb640 1 -- 172.21.15.174:0/1881644244 <== mon.2 v2:172.21.15.174:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v776) v1 ==== 217+0+0 (secure 0 0 0) 0x7fa71807c440 con 0x7fa72c105430 2024-02-10T07:03:13.307 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_bytes = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa71c06da20 msgr2=0x7fa71c06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 --2- 172.21.15.174:0/1881644244 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa71c06da20 0x7fa71c06fee0 secure :-1 s=READY pgs=3473 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d60 tx=0x7fa720005f90 comp rx=0 tx=0).stop 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 msgr2=0x7fa72c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 0x7fa72c198380 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fa718009230 tx=0x7fa718004040 comp rx=0 tx=0).stop 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 shutdown_connections 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 --2- 172.21.15.174:0/1881644244 >> v2:172.21.15.174:6800/1252702393 conn(0x7fa71c06da20 0x7fa71c06fee0 unknown :-1 s=CLOSED pgs=3473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa72c1988c0 0x7fa72c1a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.311 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72c105430 0x7fa72c198380 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.312 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 --2- 172.21.15.174:0/1881644244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa72c100360 0x7fa72c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.312 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 >> 172.21.15.174:0/1881644244 conn(0x7fa72c077250 msgr2=0x7fa72c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:13.312 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.309+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 shutdown_connections 2024-02-10T07:03:13.312 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.310+0000 7fa7330ce640 1 -- 172.21.15.174:0/1881644244 wait complete. 2024-02-10T07:03:13.322 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool set-quota 37577e97-8613-4d15-a71c-97ee4e569923 max_objects 0 2024-02-10T07:03:13.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 -- 172.21.15.174:0/3744335785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 msgr2=0x7f1164100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:13.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 --2- 172.21.15.174:0/3744335785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164100760 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f1160009930 tx=0x7f116002bc30 comp rx=0 tx=0).stop 2024-02-10T07:03:13.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 -- 172.21.15.174:0/3744335785 shutdown_connections 2024-02-10T07:03:13.412 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 --2- 172.21.15.174:0/3744335785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1164105450 0x7f116410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 --2- 172.21.15.174:0/3744335785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164100d30 0x7f1164104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 --2- 172.21.15.174:0/3744335785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164100760 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 -- 172.21.15.174:0/3744335785 >> 172.21.15.174:0/3744335785 conn(0x7f1164077310 msgr2=0x7f1164077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 -- 172.21.15.174:0/3744335785 shutdown_connections 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 -- 172.21.15.174:0/3744335785 wait complete. 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 Processor -- start 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 -- start start 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.409+0000 7f116c058640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164105450 0x7f1164198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11641988e0 0x7f11641a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11641a3560 con 0x7f1164100380 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11641a36d0 con 0x7f1164105450 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11641a3890 con 0x7f11641988e0 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:13.413 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40640/0 (socket says 172.21.15.174:40640) 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 -- 172.21.15.174:0/2071251632 learned_addr learned my addr 172.21.15.174:0/2071251632 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116a5f5640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11641988e0 0x7f11641a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f11695f3640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164105450 0x7f1164198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 -- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11641988e0 msgr2=0x7f11641a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11641988e0 0x7f11641a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 -- 172.21.15.174:0/2071251632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164105450 msgr2=0x7f1164198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164105450 0x7f1164198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c009840 con 0x7f1164100380 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f11695f3640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164105450 0x7f1164198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116a5f5640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11641988e0 0x7f11641a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f1169df4640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164197e10 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f1160000c00 tx=0x7f11600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11600044c0 con 0x7f1164100380 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7f1160003aa0 con 0x7f1164100380 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160005420 con 0x7f1164100380 2024-02-10T07:03:13.414 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1160009560 con 0x7f1164100380 2024-02-10T07:03:13.415 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.410+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11641a3e90 con 0x7f1164100380 2024-02-10T07:03:13.415 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.411+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1164068560 con 0x7f1164100380 2024-02-10T07:03:13.416 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.412+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7f11600035f0 con 0x7f1164100380 2024-02-10T07:03:13.416 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.412+0000 7f115affd640 1 --2- 172.21.15.174:0/2071251632 >> v2:172.21.15.174:6800/1252702393 conn(0x7f115006d9f0 0x7f115006feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:13.416 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.412+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(776..776 src has 1..776) v4 ==== 8231+0+0 (secure 0 0 0) 0x7f11600b04c0 con 0x7f1164100380 2024-02-10T07:03:13.416 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.412+0000 7f11695f3640 1 --2- 172.21.15.174:0/2071251632 >> v2:172.21.15.174:6800/1252702393 conn(0x7f115006d9f0 0x7f115006feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:13.416 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.412+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=777}) v3 -- 0x7f11500797e0 con 0x7f1164100380 2024-02-10T07:03:13.416 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.412+0000 7f11695f3640 1 --2- 172.21.15.174:0/2071251632 >> v2:172.21.15.174:6800/1252702393 conn(0x7f115006d9f0 0x7f115006feb0 secure :-1 s=READY pgs=3474 cs=0 l=1 rev1=1 crypto rx=0x7f116406a4a0 tx=0x7f1154005e50 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:13.418 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.417+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1160043060 con 0x7f1164100380 2024-02-10T07:03:13.597 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:13.595+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f1164100d30 con 0x7f1164100380 2024-02-10T07:03:13.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:03:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:13 smithi184 ceph-mon[33513]: osdmap e775: 8 total, 8 up, 8 in 2024-02-10T07:03:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:13 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/1881644244' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:13.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:13 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:13.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[29878]: osdmap e775: 8 total, 8 up, 8 in 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/1881644244' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[39523]: osdmap e775: 8 total, 8 up, 8 in 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/1881644244' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:13.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:13 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T07:03:14.303 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:14.301+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v777) v1 ==== 221+0+0 (secure 0 0 0) 0x7f116007d0e0 con 0x7f1164100380 2024-02-10T07:03:14.303 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:14.302+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 8 ==== osd_map(777..777 src has 1..777) v4 ==== 628+0+0 (secure 0 0 0) 0x7f1160004dc0 con 0x7f1164100380 2024-02-10T07:03:14.303 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:14.302+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=778}) v3 -- 0x7f115007a700 con 0x7f1164100380 2024-02-10T07:03:14.417 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:14.416+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f1164100f40 con 0x7f1164100380 2024-02-10T07:03:14.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:14 smithi184 ceph-mon[33513]: pgmap v1261: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:14.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:14 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:03:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:14 smithi184 ceph-mon[33513]: osdmap e776: 8 total, 8 up, 8 in 2024-02-10T07:03:14.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:14 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[29878]: pgmap v1261: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[29878]: osdmap e776: 8 total, 8 up, 8 in 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[39523]: pgmap v1261: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[39523]: osdmap e776: 8 total, 8 up, 8 in 2024-02-10T07:03:14.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:14 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:03:15.316 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.315+0000 7f115affd640 1 -- 172.21.15.174:0/2071251632 <== mon.0 v2:172.21.15.174:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 v778) v1 ==== 221+0+0 (secure 0 0 0) 0x7f1160043070 con 0x7f1164100380 2024-02-10T07:03:15.316 INFO:tasks.workunit.client.0.smithi174.stderr:set-quota max_objects = 0 for pool 37577e97-8613-4d15-a71c-97ee4e569923 2024-02-10T07:03:15.320 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.318+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 >> v2:172.21.15.174:6800/1252702393 conn(0x7f115006d9f0 msgr2=0x7f115006feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:15.320 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.318+0000 7f116c058640 1 --2- 172.21.15.174:0/2071251632 >> v2:172.21.15.174:6800/1252702393 conn(0x7f115006d9f0 0x7f115006feb0 secure :-1 s=READY pgs=3474 cs=0 l=1 rev1=1 crypto rx=0x7f116406a4a0 tx=0x7f1154005e50 comp rx=0 tx=0).stop 2024-02-10T07:03:15.320 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.319+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 msgr2=0x7f1164197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:15.320 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.319+0000 7f116c058640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164197e10 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f1160000c00 tx=0x7f11600040c0 comp rx=0 tx=0).stop 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 shutdown_connections 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 --2- 172.21.15.174:0/2071251632 >> v2:172.21.15.174:6800/1252702393 conn(0x7f115006d9f0 0x7f115006feb0 unknown :-1 s=CLOSED pgs=3474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11641988e0 0x7f11641a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1164105450 0x7f1164198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 --2- 172.21.15.174:0/2071251632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164100380 0x7f1164197e10 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 >> 172.21.15.174:0/2071251632 conn(0x7f1164077310 msgr2=0x7f11640fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 shutdown_connections 2024-02-10T07:03:15.322 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:15.320+0000 7f116c058640 1 -- 172.21.15.174:0/2071251632 wait complete. 2024-02-10T07:03:15.335 INFO:tasks.workunit.client.0.smithi174.stderr:+ sleep 30 2024-02-10T07:03:15.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:03:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:15 smithi184 ceph-mon[33513]: osdmap e777: 8 total, 8 up, 8 in 2024-02-10T07:03:15.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:15 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:03:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:03:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:15 smithi174 ceph-mon[29878]: osdmap e777: 8 total, 8 up, 8 in 2024-02-10T07:03:15.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:15 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:03:15.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:03:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:15 smithi174 ceph-mon[39523]: osdmap e777: 8 total, 8 up, 8 in 2024-02-10T07:03:15.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:15 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T07:03:16.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:16 smithi184 ceph-mon[33513]: pgmap v1264: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:16 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:03:16.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:16 smithi184 ceph-mon[33513]: osdmap e778: 8 total, 8 up, 8 in 2024-02-10T07:03:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:16 smithi174 ceph-mon[29878]: pgmap v1264: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:16 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:03:16.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:16 smithi174 ceph-mon[29878]: osdmap e778: 8 total, 8 up, 8 in 2024-02-10T07:03:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:16 smithi174 ceph-mon[39523]: pgmap v1264: 188 pgs: 188 active+clean; 857 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:16 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2071251632' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "field": "max_objects", "val": "0"}]': finished 2024-02-10T07:03:16.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:16 smithi174 ceph-mon[39523]: osdmap e778: 8 total, 8 up, 8 in 2024-02-10T07:03:17.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:03:16 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:03:17.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:17 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:17.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:17 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:17.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:17 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:18.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:17 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:03:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:03:18.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:18 smithi184 ceph-mon[33513]: pgmap v1266: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T07:03:18.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:18 smithi174 ceph-mon[29878]: pgmap v1266: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T07:03:18.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:18 smithi174 ceph-mon[39523]: pgmap v1266: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T07:03:20.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:20 smithi184 ceph-mon[33513]: pgmap v1267: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T07:03:20.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:20 smithi174 ceph-mon[29878]: pgmap v1267: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T07:03:20.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:20 smithi174 ceph-mon[39523]: pgmap v1267: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T07:03:22.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:22 smithi184 ceph-mon[33513]: pgmap v1268: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:03:22.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:22 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:22 smithi174 ceph-mon[39523]: pgmap v1268: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:03:22.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:22 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:22 smithi174 ceph-mon[29878]: pgmap v1268: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:03:22.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:22 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:24.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:24 smithi184 ceph-mon[33513]: pgmap v1269: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-10T07:03:24.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:24 smithi174 ceph-mon[29878]: pgmap v1269: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-10T07:03:24.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:24 smithi174 ceph-mon[39523]: pgmap v1269: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-10T07:03:26.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:26 smithi184 ceph-mon[33513]: pgmap v1270: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T07:03:26.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:26 smithi174 ceph-mon[29878]: pgmap v1270: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T07:03:26.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:26 smithi174 ceph-mon[39523]: pgmap v1270: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T07:03:27.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:27 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:27.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:03:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:03:27.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:27 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:27.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:27 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:28.238 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:28 smithi174 ceph-mon[29878]: pgmap v1271: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-10T07:03:28.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:27 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:03:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:03:28.239 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:28 smithi174 ceph-mon[39523]: pgmap v1271: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-10T07:03:28.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:28 smithi184 ceph-mon[33513]: pgmap v1271: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-10T07:03:30.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:30 smithi184 ceph-mon[33513]: pgmap v1272: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:30.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:30 smithi174 ceph-mon[29878]: pgmap v1272: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:30.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:30 smithi174 ceph-mon[39523]: pgmap v1272: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:32.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:32 smithi184 ceph-mon[33513]: pgmap v1273: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:32.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:32 smithi174 ceph-mon[29878]: pgmap v1273: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:32.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:32 smithi174 ceph-mon[39523]: pgmap v1273: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:34.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:34 smithi184 ceph-mon[33513]: pgmap v1274: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:34.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:34 smithi174 ceph-mon[29878]: pgmap v1274: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:34.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:34 smithi174 ceph-mon[39523]: pgmap v1274: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:36.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:36 smithi184 ceph-mon[33513]: pgmap v1275: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:36.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:36 smithi174 ceph-mon[29878]: pgmap v1275: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:36.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:36 smithi174 ceph-mon[39523]: pgmap v1275: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:37.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:37 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:37.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:03:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:03:37.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:37 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:37.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:37 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:38 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:38.239 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:38 smithi174 ceph-mon[29878]: pgmap v1276: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:38.239 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:37 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:03:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:03:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:38 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:38.240 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:38 smithi174 ceph-mon[39523]: pgmap v1276: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:38 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:03:38.372 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:38 smithi184 ceph-mon[33513]: pgmap v1276: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:40.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:40 smithi184 ceph-mon[33513]: pgmap v1277: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:40.739 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:40 smithi174 ceph-mon[29878]: pgmap v1277: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:40.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:40 smithi174 ceph-mon[39523]: pgmap v1277: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:42.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:42 smithi184 ceph-mon[33513]: pgmap v1278: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:42.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:42 smithi174 ceph-mon[29878]: pgmap v1278: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:42.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:42 smithi174 ceph-mon[39523]: pgmap v1278: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:44.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:44 smithi184 ceph-mon[33513]: pgmap v1279: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:44.738 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:44 smithi174 ceph-mon[29878]: pgmap v1279: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:44.739 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:44 smithi174 ceph-mon[39523]: pgmap v1279: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:45.336 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool delete 37577e97-8613-4d15-a71c-97ee4e569923 37577e97-8613-4d15-a71c-97ee4e569923 --yes-i-really-really-mean-it 2024-02-10T07:03:45.431 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 -- 172.21.15.174:0/2561926061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c106530 msgr2=0x7ff57c10e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:45.431 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 --2- 172.21.15.174:0/2561926061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c106530 0x7ff57c10e100 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7ff574009be0 tx=0x7ff57402bef0 comp rx=0 tx=0).stop 2024-02-10T07:03:45.431 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 -- 172.21.15.174:0/2561926061 shutdown_connections 2024-02-10T07:03:45.431 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 --2- 172.21.15.174:0/2561926061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c106530 0x7ff57c10e100 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:45.431 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 --2- 172.21.15.174:0/2561926061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c101e10 0x7ff57c105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 --2- 172.21.15.174:0/2561926061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57c101460 0x7ff57c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 -- 172.21.15.174:0/2561926061 >> 172.21.15.174:0/2561926061 conn(0x7ff57c0fccc0 msgr2=0x7ff57c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 -- 172.21.15.174:0/2561926061 shutdown_connections 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 -- 172.21.15.174:0/2561926061 wait complete. 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.428+0000 7ff583ac9640 1 Processor -- start 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 -- start start 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 0x7ff57c073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c106530 0x7ff57c073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57c073fb0 0x7ff57c1a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:45.432 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1a56f0 con 0x7ff57c106530 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1a5860 con 0x7ff57c073fb0 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff583ac9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1a5a20 con 0x7ff57c101460 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 0x7ff57c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581064640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c106530 0x7ff57c073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 0x7ff57c073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:35044/0 (socket says 172.21.15.174:35044) 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 -- 172.21.15.174:0/3142810504 learned_addr learned my addr 172.21.15.174:0/3142810504 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff582066640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57c073fb0 0x7ff57c1a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 -- 172.21.15.174:0/3142810504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57c073fb0 msgr2=0x7ff57c1a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57c073fb0 0x7ff57c1a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 -- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c106530 msgr2=0x7ff57c073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c106530 0x7ff57c073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009560 con 0x7ff57c101460 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581064640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c106530 0x7ff57c073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:03:45.433 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff581865640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 0x7ff57c073530 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7ff57800d770 tx=0x7ff57800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.429+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578009c30 con 0x7ff57c101460 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.430+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7ff57800de80 con 0x7ff57c101460 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.430+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574009840 con 0x7ff57c101460 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.430+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578004d20 con 0x7ff57c101460 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.430+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c1a6080 con 0x7ff57c101460 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.430+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c068110 con 0x7ff57c101460 2024-02-10T07:03:45.434 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.431+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7ff578004e80 con 0x7ff57c101460 2024-02-10T07:03:45.435 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.431+0000 7ff572ffd640 1 --2- 172.21.15.174:0/3142810504 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff56806da20 0x7ff56806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:45.435 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.431+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(778..778 src has 1..778) v4 ==== 8231+0+0 (secure 0 0 0) 0x7ff57808d590 con 0x7ff57c101460 2024-02-10T07:03:45.435 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.431+0000 7ff581064640 1 --2- 172.21.15.174:0/3142810504 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff56806da20 0x7ff56806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:45.435 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.432+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=779}) v3 -- 0x7ff568079890 con 0x7ff57c101460 2024-02-10T07:03:45.435 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.432+0000 7ff581064640 1 --2- 172.21.15.174:0/3142810504 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff56806da20 0x7ff56806fee0 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074fc0 tx=0x7ff56c009260 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:45.436 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.436+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff578017020 con 0x7ff57c101460 2024-02-10T07:03:45.611 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:45.611+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff57c101e10 con 0x7ff57c101460 2024-02-10T07:03:46.041 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.039+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 7 ==== osd_map(779..779 src has 1..779) v4 ==== 296+0+0 (secure 0 0 0) 0x7ff57805e5b0 con 0x7ff57c101460 2024-02-10T07:03:46.041 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.039+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7ff56807a0c0 con 0x7ff57c101460 2024-02-10T07:03:46.044 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.044+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]=0 pool '37577e97-8613-4d15-a71c-97ee4e569923' removed v779) v1 ==== 248+0+0 (secure 0 0 0) 0x7ff578020070 con 0x7ff57c101460 2024-02-10T07:03:46.152 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.152+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff57c110a20 con 0x7ff57c101460 2024-02-10T07:03:46.153 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.153+0000 7ff572ffd640 1 -- 172.21.15.174:0/3142810504 <== mon.2 v2:172.21.15.174:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]=0 pool '37577e97-8613-4d15-a71c-97ee4e569923' does not exist v779) v1 ==== 255+0+0 (secure 0 0 0) 0x7ff5780046c0 con 0x7ff57c101460 2024-02-10T07:03:46.153 INFO:tasks.workunit.client.0.smithi174.stderr:pool '37577e97-8613-4d15-a71c-97ee4e569923' does not exist 2024-02-10T07:03:46.155 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff56806da20 msgr2=0x7ff56806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:46.155 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 --2- 172.21.15.174:0/3142810504 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff56806da20 0x7ff56806fee0 secure :-1 s=READY pgs=3475 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074fc0 tx=0x7ff56c009260 comp rx=0 tx=0).stop 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 msgr2=0x7ff57c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 0x7ff57c073530 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7ff57800d770 tx=0x7ff57800dc30 comp rx=0 tx=0).stop 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 shutdown_connections 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 --2- 172.21.15.174:0/3142810504 >> v2:172.21.15.174:6800/1252702393 conn(0x7ff56806da20 0x7ff56806fee0 unknown :-1 s=CLOSED pgs=3475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff57c073fb0 0x7ff57c1a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c106530 0x7ff57c073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 --2- 172.21.15.174:0/3142810504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c101460 0x7ff57c073530 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 >> 172.21.15.174:0/3142810504 conn(0x7ff57c0fccc0 msgr2=0x7ff57c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 shutdown_connections 2024-02-10T07:03:46.156 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.155+0000 7ff583ac9640 1 -- 172.21.15.174:0/3142810504 wait complete. 2024-02-10T07:03:46.164 INFO:tasks.workunit.client.0.smithi174.stderr:+ ceph osd pool delete fd43c8b8-70b8-4fd9-961b-3c27c45d6680 fd43c8b8-70b8-4fd9-961b-3c27c45d6680 --yes-i-really-really-mean-it 2024-02-10T07:03:46.250 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 -- 172.21.15.174:0/2254565673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101e20 msgr2=0x7fcd20105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:46.250 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 --2- 172.21.15.174:0/2254565673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101e20 0x7fcd20105e60 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009930 tx=0x7fcd1002bc30 comp rx=0 tx=0).stop 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 -- 172.21.15.174:0/2254565673 shutdown_connections 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 --2- 172.21.15.174:0/2254565673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20106540 0x7fcd2010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 --2- 172.21.15.174:0/2254565673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101e20 0x7fcd20105e60 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 --2- 172.21.15.174:0/2254565673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd20101470 0x7fcd20101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 -- 172.21.15.174:0/2254565673 >> 172.21.15.174:0/2254565673 conn(0x7fcd200fcd30 msgr2=0x7fcd200ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 -- 172.21.15.174:0/2254565673 shutdown_connections 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.246+0000 7fcd27110640 1 -- 172.21.15.174:0/2254565673 wait complete. 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 Processor -- start 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 -- start start 2024-02-10T07:03:46.251 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 0x7fcd20073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd20106540 0x7fcd20073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20073fb0 0x7fcd201a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201a5810 con 0x7fcd20101470 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201a5980 con 0x7fcd20106540 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201a5b40 con 0x7fcd20073fb0 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 0x7fcd20073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd256ad640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20073fb0 0x7fcd201a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 0x7fcd20073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:35616/0 (socket says 172.21.15.174:35616) 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 -- 172.21.15.174:0/2627220754 learned_addr learned my addr 172.21.15.174:0/2627220754 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-02-10T07:03:46.252 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd17fff640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd20106540 0x7fcd20073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 -- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20073fb0 msgr2=0x7fcd201a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20073fb0 0x7fcd201a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 -- 172.21.15.174:0/2627220754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd20106540 msgr2=0x7fcd20073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd20106540 0x7fcd20073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18009840 con 0x7fcd20101470 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd256ad640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20073fb0 0x7fcd201a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd24eac640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 0x7fcd20073530 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c890 tx=0x7fcd1c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c004220 con 0x7fcd20101470 2024-02-10T07:03:46.253 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1093+0+0 (secure 0 0 0) 0x7fcd1c007e60 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10009560 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.247+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0047a0 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.248+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd201a6170 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.249+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd20068560 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.249+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) v1 ==== 91430+0+0 (secure 0 0 0) 0x7fcd1c0079b0 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.249+0000 7fcd15ffb640 1 --2- 172.21.15.174:0/2627220754 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcd0806da20 0x7fcd0806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.249+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 5 ==== osd_map(779..779 src has 1..779) v4 ==== 7843+0+0 (secure 0 0 0) 0x7fcd1c08d720 con 0x7fcd20101470 2024-02-10T07:03:46.254 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.249+0000 7fcd17fff640 1 --2- 172.21.15.174:0/2627220754 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcd0806da20 0x7fcd0806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-10T07:03:46.255 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.250+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7fcd08078ff0 con 0x7fcd20101470 2024-02-10T07:03:46.255 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.250+0000 7fcd17fff640 1 --2- 172.21.15.174:0/2627220754 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcd0806da20 0x7fcd0806fee0 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009900 tx=0x7fcd10005fd0 comp rx=0 tx=0).ready entity=mgr.14568 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T07:03:46.255 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.254+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd1c05a510 con 0x7fcd20101470 2024-02-10T07:03:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:46 smithi184 ceph-mon[33513]: pgmap v1280: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:46 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3142810504' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:46.373 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:46 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:46.426 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:46.426+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd20101e20 con 0x7fcd20101470 2024-02-10T07:03:46.488 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:46 smithi174 ceph-mon[29878]: pgmap v1280: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:46 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3142810504' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:46.489 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:46 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:46 smithi174 ceph-mon[39523]: pgmap v1280: 188 pgs: 188 active+clean; 868 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:03:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:46 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3142810504' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:46.489 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:46 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.046 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.046+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]=0 pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' removed v780) v1 ==== 248+0+0 (secure 0 0 0) 0x7fcd1c05a0a0 con 0x7fcd20101470 2024-02-10T07:03:47.048 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.048+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 8 ==== osd_map(780..780 src has 1..780) v4 ==== 296+0+0 (secure 0 0 0) 0x7fcd1c004480 con 0x7fcd20101470 2024-02-10T07:03:47.049 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.048+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7fcd08079c90 con 0x7fcd20101470 2024-02-10T07:03:47.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:03:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug there is no tcmu-runner data available 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: osdmap e779: 8 total, 8 up, 8 in 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/3142810504' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:03:47.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:47 smithi184 ceph-mon[33513]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:47.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.164+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd201109f0 con 0x7fcd20101470 2024-02-10T07:03:47.165 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.165+0000 7fcd15ffb640 1 -- 172.21.15.174:0/2627220754 <== mon.0 v2:172.21.15.174:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]=0 pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' does not exist v780) v1 ==== 255+0+0 (secure 0 0 0) 0x7fcd1c05a280 con 0x7fcd20101470 2024-02-10T07:03:47.165 INFO:tasks.workunit.client.0.smithi174.stderr:pool 'fd43c8b8-70b8-4fd9-961b-3c27c45d6680' does not exist 2024-02-10T07:03:47.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcd0806da20 msgr2=0x7fcd0806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:47.167 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 --2- 172.21.15.174:0/2627220754 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcd0806da20 0x7fcd0806fee0 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009900 tx=0x7fcd10005fd0 comp rx=0 tx=0).stop 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 msgr2=0x7fcd20073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 0x7fcd20073530 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c890 tx=0x7fcd1c00cc90 comp rx=0 tx=0).stop 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 shutdown_connections 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 --2- 172.21.15.174:0/2627220754 >> v2:172.21.15.174:6800/1252702393 conn(0x7fcd0806da20 0x7fcd0806fee0 unknown :-1 s=CLOSED pgs=3476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd20073fb0 0x7fcd201a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd20106540 0x7fcd20073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 --2- 172.21.15.174:0/2627220754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd20101470 0x7fcd20073530 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 >> 172.21.15.174:0/2627220754 conn(0x7fcd200fcd30 msgr2=0x7fcd200fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 shutdown_connections 2024-02-10T07:03:47.168 INFO:tasks.workunit.client.0.smithi174.stderr:2024-02-10T07:03:47.167+0000 7fcd27110640 1 -- 172.21.15.174:0/2627220754 wait complete. 2024-02-10T07:03:47.176 INFO:tasks.workunit.client.0.smithi174.stdout:OK 2024-02-10T07:03:47.177 INFO:tasks.workunit.client.0.smithi174.stderr:+ echo OK 2024-02-10T07:03:47.178 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-10T07:03:47.178 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-10T07:03:47.248 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-10T07:03:47.249 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: osdmap e779: 8 total, 8 up, 8 in 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/3142810504' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[29878]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:03:47.313 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: osdmap e779: 8 total, 8 up, 8 in 2024-02-10T07:03:47.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/3142810504' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "37577e97-8613-4d15-a71c-97ee4e569923", "pool2": "37577e97-8613-4d15-a71c-97ee4e569923", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:47.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:03:47.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-mon[39523]: from='client.24412 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:03:47.562 DEBUG:teuthology.parallel:result is None 2024-02-10T07:03:47.563 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T07:03:47.629 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T07:03:47.630 DEBUG:teuthology.orchestra.run.smithi174:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-10T07:03:47.699 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T07:03:47.699 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-10T07:03:47.710 INFO:tasks.cephadm:Teardown begin 2024-02-10T07:03:47.710 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:03:47.766 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:03:47.796 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-10T07:03:47.797 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:03:47.822 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:03:47.852 INFO:tasks.cephadm:Stopping all daemons... 2024-02-10T07:03:47.852 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-10T07:03:47.852 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a 2024-02-10T07:03:47.886 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:47 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: ::ffff:172.21.15.184 - - [10/Feb/2024:07:03:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: osdmap e780: 8 total, 8 up, 8 in 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: pgmap v1283: 164 pgs: 164 active+clean; 582 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:03:48.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:48 smithi184 ceph-mon[33513]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:47 smithi174 systemd[1]: Stopping Ceph mon.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: osdmap e780: 8 total, 8 up, 8 in 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: pgmap v1283: 164 pgs: 164 active+clean; 582 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[29878]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:03:48.179 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: osdmap e780: 8 total, 8 up, 8 in 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: from='client.? 172.21.15.174:0/2627220754' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "pool2": "fd43c8b8-70b8-4fd9-961b-3c27c45d6680", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: pgmap v1283: 164 pgs: 164 active+clean; 582 KiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:03:48.180 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-mon[39523]: from='mgr.14568 172.21.15.174:0/595242128' entity='mgr.y' 2024-02-10T07:03:48.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a[29855]: 2024-02-10T07:03:48.178+0000 7fba126b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:03:48.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a[29855]: 2024-02-10T07:03:48.178+0000 7fba126b4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-10T07:03:48.491 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 podman[151896]: 2024-02-10 07:03:48.250093782 +0000 UTC m=+0.200543714 container died 5b25e513001d1364c67f23b69bb4f8411ba20be74c137dba63183dd792e31412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, RELEASE=main-290212a, ceph=True, org.label-schema.build-date=20240131) 2024-02-10T07:03:48.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:48] ENGINE Bus STOPPING 2024-02-10T07:03:48.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:03:48.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:48] ENGINE Bus STOPPED 2024-02-10T07:03:48.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:48] ENGINE Bus STARTING 2024-02-10T07:03:48.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:48] ENGINE Serving on http://:::9283 2024-02-10T07:03:48.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:48 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:48] ENGINE Bus STARTED 2024-02-10T07:03:48.988 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 podman[151896]: 2024-02-10 07:03:48.717175452 +0000 UTC m=+0.667625381 container cleanup 5b25e513001d1364c67f23b69bb4f8411ba20be74c137dba63183dd792e31412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:03:48.989 INFO:journalctl@ceph.mon.a.smithi174.stdout:Feb 10 07:03:48 smithi174 bash[151896]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-a 2024-02-10T07:03:49.185 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.a.service' 2024-02-10T07:03:49.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:03:49.266 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-10T07:03:49.266 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-10T07:03:49.266 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.c 2024-02-10T07:03:49.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:49 smithi174 systemd[1]: Stopping Ceph mon.c for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:03:49.950 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c[39500]: 2024-02-10T07:03:49.590+0000 7fea908fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:03:49.950 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c[39500]: 2024-02-10T07:03:49.590+0000 7fea908fe700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-10T07:03:49.950 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:49 smithi174 podman[152042]: 2024-02-10 07:03:49.652677719 +0000 UTC m=+0.168625305 container died 564f3843f9f0df60015ab92b343714c73f3db2b0f74a87e207568cda93fd934a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-10T07:03:49.950 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:49] ENGINE Bus STOPPING 2024-02-10T07:03:49.950 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:03:49.950 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:49] ENGINE Bus STOPPED 2024-02-10T07:03:49.951 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:49] ENGINE Bus STARTING 2024-02-10T07:03:49.951 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:49] ENGINE Serving on http://:::9283 2024-02-10T07:03:49.951 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:49 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: [10/Feb/2024:07:03:49] ENGINE Bus STARTED 2024-02-10T07:03:50.200 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:50 smithi174 podman[152042]: 2024-02-10 07:03:50.040832726 +0000 UTC m=+0.556780315 container cleanup 564f3843f9f0df60015ab92b343714c73f3db2b0f74a87e207568cda93fd934a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c, io.buildah.version=1.33.2, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T07:03:50.200 INFO:journalctl@ceph.mon.c.smithi174.stdout:Feb 10 07:03:50 smithi174 bash[152042]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-c 2024-02-10T07:03:50.414 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.c.service' 2024-02-10T07:03:50.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:03:50.452 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-10T07:03:50.452 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-10T07:03:50.452 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.b 2024-02-10T07:03:50.757 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:50 smithi184 systemd[1]: Stopping Ceph mon.b for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:03:51.122 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:50 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b[33477]: 2024-02-10T07:03:50.756+0000 7f69076d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:03:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:50 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b[33477]: 2024-02-10T07:03:50.756+0000 7f69076d3700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-10T07:03:51.123 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:50 smithi184 podman[72232]: 2024-02-10 07:03:50.940794895 +0000 UTC m=+0.290386107 container died 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.schema-version=1.0) 2024-02-10T07:03:51.622 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:51 smithi184 podman[72232]: 2024-02-10 07:03:51.462679137 +0000 UTC m=+0.812270391 container cleanup 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:03:51.623 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:51 smithi184 bash[72232]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b 2024-02-10T07:03:51.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.b.service' 2024-02-10T07:03:51.951 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:51 smithi184 podman[72251]: 2024-02-10 07:03:51.681347117 +0000 UTC m=+0.741322019 container remove 17970352cfe8d40b10f86d89439369f16e44e3953e4fe279fc2686cfb447cdbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mon-b, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2) 2024-02-10T07:03:51.951 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:51 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-10T07:03:51.951 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:51 smithi184 systemd[1]: Stopped Ceph mon.b for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:03:51.951 INFO:journalctl@ceph.mon.b.smithi184.stdout:Feb 10 07:03:51 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@mon.b.service: Consumed 25.718s CPU time. 2024-02-10T07:03:51.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:03:51.975 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-10T07:03:51.975 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-10T07:03:51.975 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y 2024-02-10T07:03:52.298 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:52 smithi174 systemd[1]: Stopping Ceph mgr.y for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:03:52.620 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T07:03:52.296+0000 7f867d7b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:03:52.621 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:52 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y[30130]: 2024-02-10T07:03:52.296+0000 7f867d7b4700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-10T07:03:52.621 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:52 smithi174 podman[152184]: 2024-02-10 07:03:52.329131985 +0000 UTC m=+0.167138297 container died 28c10ca2cfc84b3391e8be5298b088d775e21b24d3dbd1361e83898b123fd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-290212a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-10T07:03:52.962 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:52 smithi174 podman[152184]: 2024-02-10 07:03:52.694696562 +0000 UTC m=+0.532702873 container cleanup 28c10ca2cfc84b3391e8be5298b088d775e21b24d3dbd1361e83898b123fd074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.2) 2024-02-10T07:03:52.962 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Feb 10 07:03:52 smithi174 bash[152184]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-y 2024-02-10T07:03:53.117 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.y.service' 2024-02-10T07:03:53.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:03:53.154 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-10T07:03:53.154 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-10T07:03:53.154 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.x 2024-02-10T07:03:53.518 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 07:03:53 smithi184 systemd[1]: Stopping Ceph mgr.x for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:03:53.802 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 07:03:53 smithi184 podman[72365]: 2024-02-10 07:03:53.518280493 +0000 UTC m=+0.166993629 container died eaa1f932db7adec413f9efd581e7352dc86b40fff6532a780096b86caf28dcdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=main-290212a, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-02-10T07:03:54.122 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 07:03:53 smithi184 podman[72365]: 2024-02-10 07:03:53.892198272 +0000 UTC m=+0.540911404 container cleanup eaa1f932db7adec413f9efd581e7352dc86b40fff6532a780096b86caf28dcdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-10T07:03:54.122 INFO:journalctl@ceph.mgr.x.smithi184.stdout:Feb 10 07:03:53 smithi184 bash[72365]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-mgr-x 2024-02-10T07:03:54.303 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@mgr.x.service' 2024-02-10T07:03:54.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:03:54.341 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-10T07:03:54.341 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-10T07:03:54.341 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.0 2024-02-10T07:03:54.651 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:03:54 smithi174 systemd[1]: Stopping Ceph osd.0 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:03:54.988 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:03:54 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0[43919]: 2024-02-10T07:03:54.650+0000 7f54f71d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:03:54.988 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:03:54 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0[43919]: 2024-02-10T07:03:54.650+0000 7f54f71d9700 -1 osd.0 780 *** Got signal Terminated *** 2024-02-10T07:03:54.989 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:03:54 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0[43919]: 2024-02-10T07:03:54.650+0000 7f54f71d9700 -1 osd.0 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:00.395 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:00 smithi174 podman[152316]: 2024-02-10 07:04:00.091046596 +0000 UTC m=+5.553553789 container died 26c39250da338ded124b6b91a299e58a62df6def47496fa83365767565c9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-10T07:04:00.739 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:00 smithi174 podman[152316]: 2024-02-10 07:04:00.440563707 +0000 UTC m=+5.903070896 container cleanup 26c39250da338ded124b6b91a299e58a62df6def47496fa83365767565c9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-290212a, ceph=True) 2024-02-10T07:04:00.739 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:00 smithi174 bash[152316]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0 2024-02-10T07:04:00.739 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:00 smithi174 podman[152340]: 2024-02-10 07:04:00.608275865 +0000 UTC m=+0.510181684 container remove 26c39250da338ded124b6b91a299e58a62df6def47496fa83365767565c9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-10T07:04:01.311 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:01 smithi174 podman[152435]: 2024-02-10 07:04:00.953202286 +0000 UTC m=+0.017536818 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:01.311 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:01 smithi174 podman[152435]: 2024-02-10 07:04:01.067537256 +0000 UTC m=+0.131871779 container create 74e80aa8bdf96d63c8ce101f6408da47a1babdccf4a856ee03a9879ec2f4039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0-deactivate, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-02-10T07:04:01.607 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:01 smithi174 podman[152435]: 2024-02-10 07:04:01.311595848 +0000 UTC m=+0.375930371 container init 74e80aa8bdf96d63c8ce101f6408da47a1babdccf4a856ee03a9879ec2f4039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0-deactivate, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-10T07:04:01.607 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:01 smithi174 podman[152435]: 2024-02-10 07:04:01.317264407 +0000 UTC m=+0.381598934 container start 74e80aa8bdf96d63c8ce101f6408da47a1babdccf4a856ee03a9879ec2f4039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-10T07:04:01.607 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:01 smithi174 podman[152435]: 2024-02-10 07:04:01.41134056 +0000 UTC m=+0.475675084 container attach 74e80aa8bdf96d63c8ce101f6408da47a1babdccf4a856ee03a9879ec2f4039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-02-10T07:04:01.988 INFO:journalctl@ceph.osd.0.smithi174.stdout:Feb 10 07:04:01 smithi174 podman[152435]: 2024-02-10 07:04:01.607604405 +0000 UTC m=+0.671938937 container died 74e80aa8bdf96d63c8ce101f6408da47a1babdccf4a856ee03a9879ec2f4039e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-02-10T07:04:02.328 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.0.service' 2024-02-10T07:04:02.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:02.377 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-10T07:04:02.377 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-10T07:04:02.377 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.1 2024-02-10T07:04:02.600 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:02 smithi174 systemd[1]: Stopping Ceph osd.1 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:02.989 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:02 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1[50156]: 2024-02-10T07:04:02.721+0000 7f4c74454700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:02.989 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:02 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1[50156]: 2024-02-10T07:04:02.721+0000 7f4c74454700 -1 osd.1 780 *** Got signal Terminated *** 2024-02-10T07:04:02.989 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:02 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1[50156]: 2024-02-10T07:04:02.721+0000 7f4c74454700 -1 osd.1 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:08.511 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:08 smithi174 podman[152575]: 2024-02-10 07:04:08.239036123 +0000 UTC m=+5.669457018 container died fb32e4c8da8b30cacd8df153caaeb421087e10fd005ff188447568c5e66dcd2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=main-290212a) 2024-02-10T07:04:08.877 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:08 smithi174 podman[152575]: 2024-02-10 07:04:08.587410341 +0000 UTC m=+6.017831236 container cleanup fb32e4c8da8b30cacd8df153caaeb421087e10fd005ff188447568c5e66dcd2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:04:08.877 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:08 smithi174 bash[152575]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1 2024-02-10T07:04:09.198 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:08 smithi174 podman[152608]: 2024-02-10 07:04:08.877335896 +0000 UTC m=+0.636506296 container remove fb32e4c8da8b30cacd8df153caaeb421087e10fd005ff188447568c5e66dcd2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-02-10T07:04:09.489 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:09 smithi174 podman[152702]: 2024-02-10 07:04:09.100789413 +0000 UTC m=+0.018043978 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:09.489 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:09 smithi174 podman[152702]: 2024-02-10 07:04:09.20299293 +0000 UTC m=+0.120247481 container create 277901b1d7c1185bdc6306205c8b904f7de8acc978543b89c920b0254a719d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, RELEASE=main-290212a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS) 2024-02-10T07:04:09.489 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:09 smithi174 podman[152702]: 2024-02-10 07:04:09.439232269 +0000 UTC m=+0.356486832 container init 277901b1d7c1185bdc6306205c8b904f7de8acc978543b89c920b0254a719d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T07:04:09.489 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:09 smithi174 podman[152702]: 2024-02-10 07:04:09.444923462 +0000 UTC m=+0.362178018 container start 277901b1d7c1185bdc6306205c8b904f7de8acc978543b89c920b0254a719d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-10T07:04:09.988 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:09 smithi174 podman[152702]: 2024-02-10 07:04:09.555751963 +0000 UTC m=+0.473006524 container attach 277901b1d7c1185bdc6306205c8b904f7de8acc978543b89c920b0254a719d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1-deactivate, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-10T07:04:09.989 INFO:journalctl@ceph.osd.1.smithi174.stdout:Feb 10 07:04:09 smithi174 podman[152702]: 2024-02-10 07:04:09.737415607 +0000 UTC m=+0.654670158 container died 277901b1d7c1185bdc6306205c8b904f7de8acc978543b89c920b0254a719d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:04:10.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.1.service' 2024-02-10T07:04:10.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:10.509 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-10T07:04:10.509 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-10T07:04:10.509 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.2 2024-02-10T07:04:10.738 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:10 smithi174 systemd[1]: Stopping Ceph osd.2 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:11.239 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:10 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2[56431]: 2024-02-10T07:04:10.847+0000 7f81b36fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:11.239 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:10 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2[56431]: 2024-02-10T07:04:10.847+0000 7f81b36fe700 -1 osd.2 780 *** Got signal Terminated *** 2024-02-10T07:04:11.239 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:10 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2[56431]: 2024-02-10T07:04:10.847+0000 7f81b36fe700 -1 osd.2 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:13.988 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:13 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:13.568Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:13.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:13 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:13.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-10T07:04:16.479 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:16 smithi174 podman[152834]: 2024-02-10 07:04:16.198582017 +0000 UTC m=+5.469543663 container died 3329848a0b2b17c6f110fcaa9c8ddae7a3d613c83030fb2a35acce2b875682c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-10T07:04:16.738 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:16 smithi174 podman[152834]: 2024-02-10 07:04:16.561202291 +0000 UTC m=+5.832163942 container cleanup 3329848a0b2b17c6f110fcaa9c8ddae7a3d613c83030fb2a35acce2b875682c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.vendor=CentOS) 2024-02-10T07:04:16.739 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:16 smithi174 bash[152834]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2 2024-02-10T07:04:17.168 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:16 smithi174 podman[152858]: 2024-02-10 07:04:16.83751012 +0000 UTC m=+0.637527465 container remove 3329848a0b2b17c6f110fcaa9c8ddae7a3d613c83030fb2a35acce2b875682c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=main-290212a, CEPH_POINT_RELEASE=) 2024-02-10T07:04:17.488 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:17 smithi174 podman[152952]: 2024-02-10 07:04:17.071392664 +0000 UTC m=+0.017910169 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:17.489 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:17 smithi174 podman[152952]: 2024-02-10 07:04:17.380044815 +0000 UTC m=+0.326562313 container create 005dd9b3d2eca6113682a8e92a78f023082ac42a31255aa742a127ba229f8701 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131) 2024-02-10T07:04:17.895 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:17 smithi174 podman[152952]: 2024-02-10 07:04:17.608269145 +0000 UTC m=+0.554786644 container init 005dd9b3d2eca6113682a8e92a78f023082ac42a31255aa742a127ba229f8701 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-02-10T07:04:17.896 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:17 smithi174 podman[152952]: 2024-02-10 07:04:17.613753691 +0000 UTC m=+0.560271192 container start 005dd9b3d2eca6113682a8e92a78f023082ac42a31255aa742a127ba229f8701 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2) 2024-02-10T07:04:17.896 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:17 smithi174 podman[152952]: 2024-02-10 07:04:17.690283088 +0000 UTC m=+0.636800587 container attach 005dd9b3d2eca6113682a8e92a78f023082ac42a31255aa742a127ba229f8701 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-10T07:04:18.193 INFO:journalctl@ceph.osd.2.smithi174.stdout:Feb 10 07:04:17 smithi174 podman[152952]: 2024-02-10 07:04:17.895927363 +0000 UTC m=+0.842444863 container died 005dd9b3d2eca6113682a8e92a78f023082ac42a31255aa742a127ba229f8701 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=main-290212a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-10T07:04:18.502 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.2.service' 2024-02-10T07:04:18.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:18.546 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-10T07:04:18.547 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-10T07:04:18.547 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.3 2024-02-10T07:04:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:04:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:04:18.290Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.174:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.174:8765: connect: connection refused" 2024-02-10T07:04:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:04:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:04:18.291Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.174:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.174:8765: connect: connection refused" 2024-02-10T07:04:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:04:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:04:18.294Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.174:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.174:8765: connect: connection refused" 2024-02-10T07:04:18.623 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:04:18 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:04:18.294Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.174:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.174:8765: connect: connection refused" 2024-02-10T07:04:18.738 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:18 smithi174 systemd[1]: Stopping Ceph osd.3 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:19.238 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:18.881+0000 7fac793fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:19.239 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:18.881+0000 7fac793fc700 -1 osd.3 780 *** Got signal Terminated *** 2024-02-10T07:04:19.239 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:18 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:18.881+0000 7fac793fc700 -1 osd.3 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:20.988 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:20 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:20.660+0000 7fac759f0700 -1 osd.3 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:56.292249+0000 front 2024-02-10T07:03:56.292226+0000 (oldest deadline 2024-02-10T07:04:20.392418+0000) 2024-02-10T07:04:21.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:21 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:21.261+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:21.988 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:21 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:21.682+0000 7fac759f0700 -1 osd.3 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:56.292249+0000 front 2024-02-10T07:03:56.292226+0000 (oldest deadline 2024-02-10T07:04:20.392418+0000) 2024-02-10T07:04:22.529 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:22 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:22.273+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:22.872 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:22 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:22.529+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:22.988 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:22 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:22.713+0000 7fac759f0700 -1 osd.3 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:56.292249+0000 front 2024-02-10T07:03:56.292226+0000 (oldest deadline 2024-02-10T07:04:20.392418+0000) 2024-02-10T07:04:23.501 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:23 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:23.245+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:23.501 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:23 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:23.211+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:23.875 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:23 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:23.501+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:23.988 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:23 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3[62937]: 2024-02-10T07:04:23.726+0000 7fac759f0700 -1 osd.3 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:56.292249+0000 front 2024-02-10T07:03:56.292226+0000 (oldest deadline 2024-02-10T07:04:20.392418+0000) 2024-02-10T07:04:23.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:23 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:23.568Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:23.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:23 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:23.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:23.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:23 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:23.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-10T07:04:24.533 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:24.205+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:24.533 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:24.252+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:24.572 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:24 smithi174 podman[153084]: 2024-02-10 07:04:24.300783895 +0000 UTC m=+5.527738653 container died b1d6aae5fdeda8f59b8a6e9c08947c0fda819f17756379a44130cfccf01ef5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-10T07:04:24.872 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:24.745+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:24.873 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:24 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:24.533+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:24.955 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:24 smithi174 podman[153084]: 2024-02-10 07:04:24.663944701 +0000 UTC m=+5.890899463 container cleanup b1d6aae5fdeda8f59b8a6e9c08947c0fda819f17756379a44130cfccf01ef5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T07:04:24.956 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:24 smithi174 bash[153084]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3 2024-02-10T07:04:25.239 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:24 smithi174 podman[153109]: 2024-02-10 07:04:24.955564355 +0000 UTC m=+0.652864228 container remove b1d6aae5fdeda8f59b8a6e9c08947c0fda819f17756379a44130cfccf01ef5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.build-date=20240131) 2024-02-10T07:04:25.488 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:25.216+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:25.488 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:25.278+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:25.549 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:25 smithi174 podman[153204]: 2024-02-10 07:04:25.15817078 +0000 UTC m=+0.018429423 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:25.550 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:25 smithi174 podman[153204]: 2024-02-10 07:04:25.256244732 +0000 UTC m=+0.116503366 container create 77177007143ae6c25444009611d53328e515784c7886f2bd94d720cc7226e8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-10T07:04:25.550 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:25 smithi174 podman[153204]: 2024-02-10 07:04:25.458365695 +0000 UTC m=+0.318624343 container init 77177007143ae6c25444009611d53328e515784c7886f2bd94d720cc7226e8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-10T07:04:25.550 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:25 smithi174 podman[153204]: 2024-02-10 07:04:25.463896116 +0000 UTC m=+0.324154755 container start 77177007143ae6c25444009611d53328e515784c7886f2bd94d720cc7226e8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-10T07:04:25.550 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:25 smithi174 podman[153204]: 2024-02-10 07:04:25.549838007 +0000 UTC m=+0.410096651 container attach 77177007143ae6c25444009611d53328e515784c7886f2bd94d720cc7226e8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3-deactivate, RELEASE=main-290212a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-10T07:04:25.872 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:25.487+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:25.873 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:25 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:25.733+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:25.989 INFO:journalctl@ceph.osd.3.smithi174.stdout:Feb 10 07:04:25 smithi174 podman[153204]: 2024-02-10 07:04:25.75296554 +0000 UTC m=+0.613224166 container died 77177007143ae6c25444009611d53328e515784c7886f2bd94d720cc7226e8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a) 2024-02-10T07:04:26.325 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.3.service' 2024-02-10T07:04:26.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:26.367 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-10T07:04:26.367 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-10T07:04:26.367 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.4 2024-02-10T07:04:26.533 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:26 smithi184 systemd[1]: Stopping Ceph osd.4 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:26.533 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:26.185+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:26.533 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:26.273+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:26.873 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:26.735+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:26.873 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:26.532+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:27.315 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:27.186+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:27.316 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:26.979+0000 7f8797eb3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:27.316 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:26.979+0000 7f8797eb3700 -1 osd.4 780 *** Got signal Terminated *** 2024-02-10T07:04:27.316 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:26 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:26.979+0000 7f8797eb3700 -1 osd.4 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:27.626 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:27.314+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:27.628 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:27.530+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:28.122 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:27 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:27.782+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:28.510 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:28 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:28.170+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:28.510 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:28 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:28.296+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:28.810 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:28 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:28.509+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:29.122 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:28 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:28.809+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:29.537 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:29.149+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:29.537 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:29.335+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:29.820 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:29.535+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:30.122 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:29 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:29.820+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:30.487 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:30 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:30.174+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:30.487 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:30 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:30.341+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:30.858 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:30 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:30.486+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:31.122 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:30 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:30.857+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:31.450 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:31 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:31.143+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:31.450 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:31 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:31.336+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:31.450 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:31 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:31.336+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:31.824 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:31 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:31.448+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:32.122 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:31 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4[38923]: 2024-02-10T07:04:31.824+0000 7f87944a7700 -1 osd.4 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:59.021022+0000 front 2024-02-10T07:03:59.020967+0000 (oldest deadline 2024-02-10T07:04:24.320889+0000) 2024-02-10T07:04:32.454 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:32 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:32.127+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:32.454 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:32 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:32.289+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:32.454 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:32 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:32.289+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:32.755 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:32 smithi184 podman[72502]: 2024-02-10 07:04:32.454521624 +0000 UTC m=+5.889675034 container died 4993678cedd8d93fc59fb0a200e3e6bda056b7f386969c4bdc6c014ebfd75958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-10T07:04:32.755 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:32 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:32.481+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:32.756 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:32 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:32.481+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:33.029 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:32 smithi184 podman[72502]: 2024-02-10 07:04:32.861807162 +0000 UTC m=+6.296960571 container cleanup 4993678cedd8d93fc59fb0a200e3e6bda056b7f386969c4bdc6c014ebfd75958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-10T07:04:33.029 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:32 smithi184 bash[72502]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4 2024-02-10T07:04:33.373 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72524]: 2024-02-10 07:04:33.029674248 +0000 UTC m=+0.578598804 container remove 4993678cedd8d93fc59fb0a200e3e6bda056b7f386969c4bdc6c014ebfd75958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-02-10T07:04:33.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:33 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:33.123+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:33.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:33 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:33.123+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:33.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:33 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:33.257+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:33.374 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:33 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:33.257+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:33.694 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:33 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:33.447+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:33.694 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:33 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:33.447+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:33.695 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72624]: 2024-02-10 07:04:33.332272144 +0000 UTC m=+0.018001569 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:33.695 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72624]: 2024-02-10 07:04:33.44721285 +0000 UTC m=+0.132942309 container create 0a80a16b93591539cb0d4e4b9ddc35664fcf4ab3784651d362f407e8938ccf43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-10T07:04:33.947 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:33 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:33.568Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:33.948 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:33 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:33.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:33.948 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:33 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:33.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-10T07:04:33.985 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72624]: 2024-02-10 07:04:33.691204681 +0000 UTC m=+0.376934105 container init 0a80a16b93591539cb0d4e4b9ddc35664fcf4ab3784651d362f407e8938ccf43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-10T07:04:33.986 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72624]: 2024-02-10 07:04:33.697156527 +0000 UTC m=+0.382885952 container start 0a80a16b93591539cb0d4e4b9ddc35664fcf4ab3784651d362f407e8938ccf43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-10T07:04:33.986 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72624]: 2024-02-10 07:04:33.782852646 +0000 UTC m=+0.468582071 container attach 0a80a16b93591539cb0d4e4b9ddc35664fcf4ab3784651d362f407e8938ccf43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-10T07:04:34.328 INFO:journalctl@ceph.osd.4.smithi184.stdout:Feb 10 07:04:33 smithi184 podman[72624]: 2024-02-10 07:04:33.986051599 +0000 UTC m=+0.671781008 container died 0a80a16b93591539cb0d4e4b9ddc35664fcf4ab3784651d362f407e8938ccf43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-10T07:04:34.328 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:34.113+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:34.328 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:34.113+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:34.329 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:34.233+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:34.329 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:34.233+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:34.611 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:34.426+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:34.611 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:34 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:34.426+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:34.639 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.4.service' 2024-02-10T07:04:34.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:34.682 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-10T07:04:34.682 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-10T07:04:34.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.5 2024-02-10T07:04:34.872 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:34 smithi184 systemd[1]: Stopping Ceph osd.5 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:35.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:35.218+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:35.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:35.218+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:35.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:35.009+0000 7f6ff92f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:35.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:35.009+0000 7f6ff92f9700 -1 osd.5 780 *** Got signal Terminated *** 2024-02-10T07:04:35.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:35.009+0000 7f6ff92f9700 -1 osd.5 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:35.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:35.156+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:35.374 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:35.156+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:35.872 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:35.384+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:35.873 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:35 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:35.384+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:36.622 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:36.125+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:36.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:36.125+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:36.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:36.345+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:36.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:36.345+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:36.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:36.229+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:36.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:36 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:36.229+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:37.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:37.103+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:37.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:37.103+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:37.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:37.274+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:37.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:37.274+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:37.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:37.274+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:37.872 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:37.375+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:37.873 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:37.375+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:37.873 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:37 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:37.375+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:38.622 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:38.127+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:38.127+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:38.301+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:38.301+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:38.301+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:38.330+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:38.330+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:38.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:38 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:38.330+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:39.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:39.137+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.279715+0000 front 2024-02-10T07:03:58.279749+0000 (oldest deadline 2024-02-10T07:04:20.579670+0000) 2024-02-10T07:04:39.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:39.137+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.581297+0000 front 2024-02-10T07:04:07.581281+0000 (oldest deadline 2024-02-10T07:04:32.281199+0000) 2024-02-10T07:04:39.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5[44899]: 2024-02-10T07:04:39.137+0000 7f6ff18e5700 -1 osd.5 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:12.281754+0000 front 2024-02-10T07:04:12.281667+0000 (oldest deadline 2024-02-10T07:04:38.181608+0000) 2024-02-10T07:04:39.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:39.286+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:39.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:39.286+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:39.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:39.286+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:39.624 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:39.377+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:39.625 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:39.377+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:39.625 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:39 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:39.377+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:40.623 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:40 smithi184 podman[72754]: 2024-02-10 07:04:40.497455732 +0000 UTC m=+5.592969500 container died 3bbc23243d0a9b252bad5dd96304055f380bc6a570b6d203d741f000284f2171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-10T07:04:40.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:40.377+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:40.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:40.377+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:40.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:40.377+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:40.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:40.291+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:40.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:40.291+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:40.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:40 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:40.291+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:41.031 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:40 smithi184 podman[72754]: 2024-02-10 07:04:40.838681778 +0000 UTC m=+5.934195545 container cleanup 3bbc23243d0a9b252bad5dd96304055f380bc6a570b6d203d741f000284f2171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:04:41.031 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:40 smithi184 bash[72754]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5 2024-02-10T07:04:41.372 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72777]: 2024-02-10 07:04:41.031662163 +0000 UTC m=+0.524330237 container remove 3bbc23243d0a9b252bad5dd96304055f380bc6a570b6d203d741f000284f2171 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-10T07:04:41.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:41.272+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:41.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:41.272+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:41.373 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:41.272+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:41.693 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:41.414+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:41.694 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:41.414+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:41.694 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:41 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:41.414+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:41.694 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72872]: 2024-02-10 07:04:41.350185535 +0000 UTC m=+0.018355315 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:41.694 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72872]: 2024-02-10 07:04:41.48278979 +0000 UTC m=+0.150959606 container create 2a29d777e6f4af3d647c7eb53e798859d38d7ed30ba27719db3a7c69e1bdc357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-10T07:04:41.694 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72872]: 2024-02-10 07:04:41.693788284 +0000 UTC m=+0.361958058 container init 2a29d777e6f4af3d647c7eb53e798859d38d7ed30ba27719db3a7c69e1bdc357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240131, RELEASE=main-290212a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-10T07:04:41.988 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72872]: 2024-02-10 07:04:41.699591952 +0000 UTC m=+0.367761727 container start 2a29d777e6f4af3d647c7eb53e798859d38d7ed30ba27719db3a7c69e1bdc357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-10T07:04:41.988 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72872]: 2024-02-10 07:04:41.768423607 +0000 UTC m=+0.436593385 container attach 2a29d777e6f4af3d647c7eb53e798859d38d7ed30ba27719db3a7c69e1bdc357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:04:42.287 INFO:journalctl@ceph.osd.5.smithi184.stdout:Feb 10 07:04:41 smithi184 podman[72872]: 2024-02-10 07:04:41.988753567 +0000 UTC m=+0.656923350 container died 2a29d777e6f4af3d647c7eb53e798859d38d7ed30ba27719db3a7c69e1bdc357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux ) 2024-02-10T07:04:42.581 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:42.399+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:42.581 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:42.399+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:42.581 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:42.399+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:42.581 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:42.307+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:42.581 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:42.307+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:42.581 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:42.307+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:42.600 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.5.service' 2024-02-10T07:04:42.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:42.643 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-10T07:04:42.643 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-10T07:04:42.643 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.6 2024-02-10T07:04:42.872 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 systemd[1]: Stopping Ceph osd.6 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:43.290 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:42.968+0000 7f37a75ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:43.290 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:42.968+0000 7f37a75ae700 -1 osd.6 780 *** Got signal Terminated *** 2024-02-10T07:04:43.290 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:42 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:42.968+0000 7f37a75ae700 -1 osd.6 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:43.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:43.436+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:43.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:43.436+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:43.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:43.436+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:43.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:43.289+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:43.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:43.289+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:43.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:43 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:43.289+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:43.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:43 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:43.569Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:43.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:43 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:43.569Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:43.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:43 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:43.570Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-10T07:04:44.622 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:44.432+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:44.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:44.432+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:44.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:44.432+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:44.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:44.320+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:44.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:44.320+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:44.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:44 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:44.320+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:45.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:45.408+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:45.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:45.408+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:45.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:45.408+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:45.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:45.337+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:45.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:45.337+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:45.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:45.337+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:45.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:45 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:45.337+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:46.622 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:46.443+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:46.443+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:46.443+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:46.443+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:23.275709+0000 front 2024-02-10T07:04:23.275633+0000 (oldest deadline 2024-02-10T07:04:45.575659+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:46.331+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:46.331+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:46.331+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:46.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:46 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:46.331+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:47.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:47.298+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:47.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:47.298+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:47.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:47.298+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:47.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:47.298+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:47.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:47.458+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:57.572526+0000 front 2024-02-10T07:03:57.572416+0000 (oldest deadline 2024-02-10T07:04:21.672341+0000) 2024-02-10T07:04:47.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:47.458+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:07.573323+0000 front 2024-02-10T07:04:07.573421+0000 (oldest deadline 2024-02-10T07:04:32.273297+0000) 2024-02-10T07:04:47.624 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:47.458+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:13.374226+0000 front 2024-02-10T07:04:13.374187+0000 (oldest deadline 2024-02-10T07:04:36.874074+0000) 2024-02-10T07:04:47.624 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:47 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6[50870]: 2024-02-10T07:04:47.458+0000 7f379fb9a700 -1 osd.6 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:23.275709+0000 front 2024-02-10T07:04:23.275633+0000 (oldest deadline 2024-02-10T07:04:45.575659+0000) 2024-02-10T07:04:48.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:48 smithi184 podman[73003]: 2024-02-10 07:04:48.464874287 +0000 UTC m=+5.600348790 container died 40bd8bb4285dac326c3fbe087ee9d1e900a33e4ff18118ae5fd4aa0a3142caa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-290212a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-10T07:04:48.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:48 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:48.339+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:48.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:48 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:48.339+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:48.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:48 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:48.339+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:48.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:48 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:48.339+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:48.969 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:48 smithi184 podman[73003]: 2024-02-10 07:04:48.849610691 +0000 UTC m=+5.985085195 container cleanup 40bd8bb4285dac326c3fbe087ee9d1e900a33e4ff18118ae5fd4aa0a3142caa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-10T07:04:48.969 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:48 smithi184 bash[73003]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6 2024-02-10T07:04:49.330 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:49 smithi184 podman[73026]: 2024-02-10 07:04:49.125890739 +0000 UTC m=+0.657165055 container remove 40bd8bb4285dac326c3fbe087ee9d1e900a33e4ff18118ae5fd4aa0a3142caa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-290212a) 2024-02-10T07:04:49.622 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:49 smithi184 podman[73128]: 2024-02-10 07:04:49.421073713 +0000 UTC m=+0.022237433 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:49.623 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:49 smithi184 podman[73128]: 2024-02-10 07:04:49.543564155 +0000 UTC m=+0.144727868 container create 1243dac1a179c815c624afdee57df39c6d30c631849ff825ab884751c387edf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-10T07:04:49.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:49 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:49.329+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:49.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:49 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:49.329+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:49.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:49 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:49.329+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:49.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:49 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:49.329+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:49.975 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:49 smithi184 podman[73128]: 2024-02-10 07:04:49.746119449 +0000 UTC m=+0.347283175 container init 1243dac1a179c815c624afdee57df39c6d30c631849ff825ab884751c387edf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6-deactivate, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-10T07:04:49.975 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:49 smithi184 podman[73128]: 2024-02-10 07:04:49.752179943 +0000 UTC m=+0.353343663 container start 1243dac1a179c815c624afdee57df39c6d30c631849ff825ab884751c387edf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-10T07:04:49.976 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:49 smithi184 podman[73128]: 2024-02-10 07:04:49.828999098 +0000 UTC m=+0.430162812 container attach 1243dac1a179c815c624afdee57df39c6d30c631849ff825ab884751c387edf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-290212a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2) 2024-02-10T07:04:50.348 INFO:journalctl@ceph.osd.6.smithi184.stdout:Feb 10 07:04:50 smithi184 podman[73128]: 2024-02-10 07:04:50.05139829 +0000 UTC m=+0.652562004 container died 1243dac1a179c815c624afdee57df39c6d30c631849ff825ab884751c387edf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:04:50.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:50 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:50.374+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:50.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:50 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:50.374+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:50.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:50 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:50.374+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:50.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:50 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:50.374+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:50.667 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.6.service' 2024-02-10T07:04:50.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:50.711 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-10T07:04:50.712 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-10T07:04:50.712 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.7 2024-02-10T07:04:51.039 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:50 smithi184 systemd[1]: Stopping Ceph osd.7 for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:51.039 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.038+0000 7f27135bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:51.348 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.038+0000 7f27135bf700 -1 osd.7 780 *** Got signal Terminated *** 2024-02-10T07:04:51.348 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.038+0000 7f27135bf700 -1 osd.7 780 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:04:51.348 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.347+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:51.348 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.347+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:51.348 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.347+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:51.348 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:51 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:51.347+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:52.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:52.362+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:52.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:52.362+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:52.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:52.362+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:52.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:52 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:52.362+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:53.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:53 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:53.322+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:53.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:53 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:53.322+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:53.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:53 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:53.322+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:53.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:53 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:53.322+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:53.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:53.570Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:53.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:53.570Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:04:53.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:04:53 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:04:53.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-10T07:04:54.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:54 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:54.356+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:54.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:54 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:54.356+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:54.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:54 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:54.356+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:54.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:54 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:54.356+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:55.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:55.313+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6803 osd.0 since back 2024-02-10T07:03:58.195887+0000 front 2024-02-10T07:03:58.195903+0000 (oldest deadline 2024-02-10T07:04:22.895711+0000) 2024-02-10T07:04:55.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:55.313+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6807 osd.1 since back 2024-02-10T07:04:02.896190+0000 front 2024-02-10T07:04:02.896200+0000 (oldest deadline 2024-02-10T07:04:31.096813+0000) 2024-02-10T07:04:55.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:55.313+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6811 osd.2 since back 2024-02-10T07:04:11.097386+0000 front 2024-02-10T07:04:11.097297+0000 (oldest deadline 2024-02-10T07:04:36.397282+0000) 2024-02-10T07:04:55.623 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:55 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7[56862]: 2024-02-10T07:04:55.313+0000 7f270bbab700 -1 osd.7 780 heartbeat_check: no reply from 172.21.15.174:6815 osd.3 since back 2024-02-10T07:04:19.298163+0000 front 2024-02-10T07:04:19.298150+0000 (oldest deadline 2024-02-10T07:04:44.498499+0000) 2024-02-10T07:04:56.624 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:56 smithi184 podman[73264]: 2024-02-10 07:04:56.359261924 +0000 UTC m=+5.421401542 container died 6a36ed3d1ad369406c4a6cb9156eb38798511e5c7aab1a100310a74d2f1af7b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-02-10T07:04:57.011 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:56 smithi184 podman[73264]: 2024-02-10 07:04:56.7516845 +0000 UTC m=+5.813824123 container cleanup 6a36ed3d1ad369406c4a6cb9156eb38798511e5c7aab1a100310a74d2f1af7b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7, RELEASE=main-290212a, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:04:57.011 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:56 smithi184 bash[73264]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7 2024-02-10T07:04:57.311 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73286]: 2024-02-10 07:04:57.011082673 +0000 UTC m=+0.643671940 container remove 6a36ed3d1ad369406c4a6cb9156eb38798511e5c7aab1a100310a74d2f1af7b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-10T07:04:57.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73382]: 2024-02-10 07:04:57.311848131 +0000 UTC m=+0.115125050 container create 5f9390bddde902813dc3e715b87fa29ec3d67143364e2d071cdb96796f2890a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T07:04:57.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73382]: 2024-02-10 07:04:57.21487596 +0000 UTC m=+0.018152886 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:04:57.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73382]: 2024-02-10 07:04:57.531025572 +0000 UTC m=+0.334302500 container init 5f9390bddde902813dc3e715b87fa29ec3d67143364e2d071cdb96796f2890a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.build-date=20240131, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-10T07:04:57.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73382]: 2024-02-10 07:04:57.536887713 +0000 UTC m=+0.340164636 container start 5f9390bddde902813dc3e715b87fa29ec3d67143364e2d071cdb96796f2890a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-10T07:04:57.622 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73382]: 2024-02-10 07:04:57.622154016 +0000 UTC m=+0.425430947 container attach 5f9390bddde902813dc3e715b87fa29ec3d67143364e2d071cdb96796f2890a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True) 2024-02-10T07:04:57.872 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:57 smithi184 podman[73382]: 2024-02-10 07:04:57.822246121 +0000 UTC m=+0.625523039 container died 5f9390bddde902813dc3e715b87fa29ec3d67143364e2d071cdb96796f2890a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-10T07:04:58.327 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.7.service' 2024-02-10T07:04:58.354 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:58 smithi184 podman[73428]: 2024-02-10 07:04:58.308556983 +0000 UTC m=+0.476010473 container remove 5f9390bddde902813dc3e715b87fa29ec3d67143364e2d071cdb96796f2890a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-10T07:04:58.354 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:58 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.7.service: Deactivated successfully. 2024-02-10T07:04:58.354 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:58 smithi184 systemd[1]: Stopped Ceph osd.7 for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:04:58.354 INFO:journalctl@ceph.osd.7.smithi184.stdout:Feb 10 07:04:58 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@osd.7.service: Consumed 41.795s CPU time. 2024-02-10T07:04:58.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:04:58.368 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-10T07:04:58.369 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-10T07:04:58.369 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@rgw.foo.a 2024-02-10T07:04:58.672 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:04:58 smithi174 systemd[1]: Stopping Ceph rgw.foo.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:04:58.988 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:04:58 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a[67900]: 2024-02-10T07:04:58.671+0000 7f90ffa6f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:04:58.989 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:04:58 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a[67900]: 2024-02-10T07:04:58.671+0000 7f910e2e0ac0 -1 shutting down 2024-02-10T07:05:03.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:03 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:05:03.571Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:05:03.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:03 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:05:03.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi174.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.174:8443: connect: connection refused" 2024-02-10T07:05:03.989 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:03 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:05:03.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi184.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.184:8443: connect: connection refused" 2024-02-10T07:05:08.989 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:08 smithi174 bash[153347]: time="2024-02-10T07:05:08Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-10T07:05:08.989 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:08 smithi174 podman[153347]: 2024-02-10 07:05:08.701565635 +0000 UTC m=+10.136553396 container died a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-10T07:05:09.346 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 podman[153347]: 2024-02-10 07:05:09.150962678 +0000 UTC m=+10.585950438 container cleanup a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:05:09.346 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 bash[153347]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a 2024-02-10T07:05:09.574 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@rgw.foo.a.service' 2024-02-10T07:05:09.600 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 podman[153384]: 2024-02-10 07:05:09.34600536 +0000 UTC m=+0.640454504 container remove a1e4a734fcac5b7d159429bdb02018d2a9e3def6a142588ae7fa9a4fae2d4491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux ) 2024-02-10T07:05:09.600 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-10T07:05:09.600 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2024-02-10T07:05:09.600 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 systemd[1]: Stopped Ceph rgw.foo.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:05:09.600 INFO:journalctl@ceph.rgw.foo.a.smithi174.stdout:Feb 10 07:05:09 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@rgw.foo.a.service: Consumed 3.770s CPU time. 2024-02-10T07:05:09.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:05:09.614 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-10T07:05:09.615 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-10T07:05:09.615 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-df845a14-c7de-11ee-95b8-87774f69a715@prometheus.a 2024-02-10T07:05:09.929 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 systemd[1]: Stopping Ceph prometheus.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:05:10.227 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.929Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.930Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.930Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.930Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a[67805]: ts=2024-02-10T07:05:09.930Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-10T07:05:10.228 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:09 smithi184 podman[73514]: 2024-02-10 07:05:09.939829547 +0000 UTC m=+0.124815958 container died d0b9bb860aa0848b72014550e31ff762bfc5d3327d1712f55e462618f102e782 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:10.568 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:10 smithi184 podman[73514]: 2024-02-10 07:05:10.334792459 +0000 UTC m=+0.519778912 container cleanup d0b9bb860aa0848b72014550e31ff762bfc5d3327d1712f55e462618f102e782 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:10.568 INFO:journalctl@ceph.prometheus.a.smithi184.stdout:Feb 10 07:05:10 smithi184 bash[73514]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T07:05:10.698 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df845a14-c7de-11ee-95b8-87774f69a715@prometheus.a.service' 2024-02-10T07:05:10.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:05:10.736 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-10T07:05:10.737 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm rm-cluster --fsid df845a14-c7de-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T07:05:10.925 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T07:05:12.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:12 smithi174 systemd[1]: Stopping Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:05:13.073 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:12 smithi174 ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a[78862]: ts=2024-02-10T07:05:12.742Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-10T07:05:13.073 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:12 smithi174 podman[153866]: 2024-02-10 07:05:12.804011702 +0000 UTC m=+0.176432101 container died b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:13.423 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 podman[153866]: 2024-02-10 07:05:13.17237368 +0000 UTC m=+0.544794079 container cleanup b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:13.423 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 bash[153866]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T07:05:13.738 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 podman[153886]: 2024-02-10 07:05:13.423270084 +0000 UTC m=+0.618263848 container remove b20e040e57059deb0c84806ca009d4a47a353bf6c5c2ea2c60cfc64488d6f2d1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:13.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 podman[153886]: 2024-02-10 07:05:13.498884618 +0000 UTC m=+0.693878399 volume remove 30ecf2cc03fbc5a696e7f095a4db779e4f82c6f44718e2b6a03d99c48aa5600d 2024-02-10T07:05:13.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-10T07:05:13.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 systemd[1]: Stopped Ceph alertmanager.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:05:13.739 INFO:journalctl@ceph.alertmanager.a.smithi174.stdout:Feb 10 07:05:13 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@alertmanager.a.service: Consumed 1.827s CPU time. 2024-02-10T07:05:14.082 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:13 smithi174 systemd[1]: Stopping Ceph node-exporter.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:05:14.360 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 podman[154004]: 2024-02-10 07:05:14.082167828 +0000 UTC m=+0.122571526 container died 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:14.686 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 podman[154004]: 2024-02-10 07:05:14.460000456 +0000 UTC m=+0.500404156 container cleanup 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:14.686 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 bash[154004]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a 2024-02-10T07:05:14.971 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 podman[154023]: 2024-02-10 07:05:14.685998436 +0000 UTC m=+0.608588747 container remove 7c458c09382f03613cda0de4c9249068509ad1ca15fa2586d0cffcb38bf329d7 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:05:14.972 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:05:14.972 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-10T07:05:14.972 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 systemd[1]: Stopped Ceph node-exporter.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:05:14.972 INFO:journalctl@ceph.node-exporter.a.smithi174.stdout:Feb 10 07:05:14 smithi174 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.a.service: Consumed 3.678s CPU time. 2024-02-10T07:05:15.507 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm rm-cluster --fsid df845a14-c7de-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T07:05:15.699 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T07:05:17.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:16 smithi184 systemd[1]: Stopping Ceph iscsi.iscsi.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:05:17.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:17 smithi184 ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a[62225]: debug Shutdown received 2024-02-10T07:05:27.534 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:27 smithi184 bash[73940]: time="2024-02-10T07:05:27Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-10T07:05:27.534 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:27 smithi184 podman[73940]: 2024-02-10 07:05:27.267054004 +0000 UTC m=+10.167413934 container died 4090fe60e7f66764eea836f50d241eb7b2d5e42b671aeaf79be9abc217c79c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, ceph=True, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:05:27.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:27 smithi184 podman[73940]: 2024-02-10 07:05:27.695811868 +0000 UTC m=+10.596171807 container cleanup 4090fe60e7f66764eea836f50d241eb7b2d5e42b671aeaf79be9abc217c79c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:05:27.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:27 smithi184 bash[73940]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a 2024-02-10T07:05:28.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:27 smithi184 podman[73963]: 2024-02-10 07:05:27.912680294 +0000 UTC m=+0.643218259 container remove 4090fe60e7f66764eea836f50d241eb7b2d5e42b671aeaf79be9abc217c79c2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-iscsi-iscsi-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-10T07:05:28.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:27 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-10T07:05:28.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:28 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-10T07:05:28.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:28 smithi184 systemd[1]: Stopped Ceph iscsi.iscsi.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:05:28.328 INFO:journalctl@ceph.iscsi.iscsi.a.smithi184.stdout:Feb 10 07:05:28 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Consumed 3.481s CPU time. 2024-02-10T07:05:29.872 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:29 smithi184 systemd[1]: Stopping Ceph grafana.a for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:05:30.178 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:29 smithi184 podman[74252]: 2024-02-10 07:05:29.875913962 +0000 UTC m=+0.124469841 container died 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, release=239.1651231664, vendor=Red Hat, Inc., version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, vcs-type=git, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-02-10T07:05:30.178 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 podman[74252]: 2024-02-10 07:05:30.178697114 +0000 UTC m=+0.427253005 container cleanup 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a, version=8.5, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , description=Ceph Grafana Container, io.openshift.expose-services=, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public) 2024-02-10T07:05:30.458 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 bash[74252]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a 2024-02-10T07:05:30.458 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 podman[74271]: 2024-02-10 07:05:30.388192393 +0000 UTC m=+0.514937355 container remove 2abb0e9cab286283c69fd0d99976d351b951a62ee90b8b27a1562d60aa458f38 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-grafana-a, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, vendor=Red Hat, Inc., io.buildah.version=1.32.0, release=239.1651231664, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, name=ubi8) 2024-02-10T07:05:30.458 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:05:30.712 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-10T07:05:30.713 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 systemd[1]: Stopped Ceph grafana.a for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:05:30.713 INFO:journalctl@ceph.grafana.a.smithi184.stdout:Feb 10 07:05:30 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@grafana.a.service: Consumed 11.107s CPU time. 2024-02-10T07:05:30.971 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:30 smithi184 systemd[1]: Stopping Ceph node-exporter.b for df845a14-c7de-11ee-95b8-87774f69a715... 2024-02-10T07:05:31.241 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:30 smithi184 podman[74391]: 2024-02-10 07:05:30.971164592 +0000 UTC m=+0.121350496 container died d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:05:31.622 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 podman[74391]: 2024-02-10 07:05:31.340676921 +0000 UTC m=+0.490862832 container cleanup d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:05:31.622 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 bash[74391]: ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b 2024-02-10T07:05:31.902 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 podman[74410]: 2024-02-10 07:05:31.625006461 +0000 UTC m=+0.658505588 container remove d6ef50193be64e4f1b684aee4c2193a4794085a447c5ff40f3b0ccfd9eea0135 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-df845a14-c7de-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:05:31.902 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:05:31.903 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-10T07:05:31.903 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 systemd[1]: Stopped Ceph node-exporter.b for df845a14-c7de-11ee-95b8-87774f69a715. 2024-02-10T07:05:31.903 INFO:journalctl@ceph.node-exporter.b.smithi184.stdout:Feb 10 07:05:31 smithi184 systemd[1]: ceph-df845a14-c7de-11ee-95b8-87774f69a715@node-exporter.b.service: Consumed 3.595s CPU time. 2024-02-10T07:05:32.437 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:05:32.474 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:05:32.503 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-10T07:05:32.505 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/remote/smithi174/crash 2024-02-10T07:05:32.506 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/crash -- . 2024-02-10T07:05:32.565 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T07:05:32.566 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-02-10T07:05:32.567 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/remote/smithi184/crash 2024-02-10T07:05:32.568 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/crash -- . 2024-02-10T07:05:32.610 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/df845a14-c7de-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T07:05:32.610 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-02-10T07:05:32.611 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-10T07:05:32.612 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T07:05:32.644 INFO:teuthology.orchestra.run.smithi174.stdout:2024-02-10T06:50:00.000246+0000 mon.a (mon.0) 2217 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T07:05:32.644 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-10T07:05:32.644 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[SEC\]' /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T07:05:32.716 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[ERR\]' /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T07:05:32.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo egrep '\[WRN\]' /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T07:05:32.860 INFO:teuthology.orchestra.run.smithi174.stdout:2024-02-10T06:50:00.000246+0000 mon.a (mon.0) 2217 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T07:05:32.860 INFO:tasks.cephadm:Compressing logs... 2024-02-10T07:05:32.860 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T07:05:32.903 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr: /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.a.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.a.log: gzip/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.audit.log: -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.1.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.3.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mgr.y.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.1.log: /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.0.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log 2024-02-10T07:05:32.930 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.cephadm.log: 89.0% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.2.log 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.6.log 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi184.stderr:: No such file or directory 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.4.log 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.7.log 2024-02-10T07:05:32.933 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.audit.log: /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.5.log 2024-02-10T07:05:32.937 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mgr.x.log 2024-02-10T07:05:32.937 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/tcmu-runner.log 2024-02-10T07:05:32.938 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T07:05:32.938 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/tcmu-runner.log: gzip -5 --verbose 67.5% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/tcmu-runner.log.gz 2024-02-10T07:05:32.938 INFO:teuthology.orchestra.run.smithi184.stderr: -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log 2024-02-10T07:05:32.938 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T07:05:32.938 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log: /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.cephadm.log: 81.8% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T07:05:32.938 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.b.log 2024-02-10T07:05:32.939 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-volume.log: 87.0% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T07:05:32.940 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log: 92.7% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T07:05:32.940 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T07:05:32.941 INFO:teuthology.orchestra.run.smithi184.stderr: 92.3% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T07:05:32.942 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T07:05:32.943 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mgr.x.log.gz 2024-02-10T07:05:32.945 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.2.log: 95.2% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T07:05:32.947 INFO:teuthology.orchestra.run.smithi184.stderr: 85.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T07:05:32.949 INFO:teuthology.orchestra.run.smithi184.stderr: 90.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T07:05:32.952 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.c.log 2024-02-10T07:05:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T07:05:32.966 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T07:05:32.968 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-10T07:05:32.974 INFO:teuthology.orchestra.run.smithi174.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T07:05:33.235 INFO:teuthology.orchestra.run.smithi174.stderr: 90.7% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mgr.y.log.gz 2024-02-10T07:05:33.886 INFO:teuthology.orchestra.run.smithi184.stderr: 93.1% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.b.log.gz 2024-02-10T07:05:34.035 INFO:teuthology.orchestra.run.smithi174.stderr: 92.7% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.c.log.gz 2024-02-10T07:05:35.140 INFO:teuthology.orchestra.run.smithi174.stderr: 92.1% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-mon.a.log.gz 2024-02-10T07:05:39.448 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.4.log.gz 2024-02-10T07:05:39.576 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.5.log.gz 2024-02-10T07:05:39.581 INFO:teuthology.orchestra.run.smithi184.stderr: 93.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.7.log.gz 2024-02-10T07:05:39.592 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.6.log.gz 2024-02-10T07:05:39.594 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-02-10T07:05:39.594 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m6.675s 2024-02-10T07:05:39.594 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m26.482s 2024-02-10T07:05:39.594 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.951s 2024-02-10T07:05:39.656 INFO:teuthology.orchestra.run.smithi174.stderr: 93.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.0.log.gz 2024-02-10T07:05:40.247 INFO:teuthology.orchestra.run.smithi174.stderr: 93.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.3.log.gz 2024-02-10T07:05:40.298 INFO:teuthology.orchestra.run.smithi174.stderr: 93.9% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.2.log.gz 2024-02-10T07:05:40.734 INFO:teuthology.orchestra.run.smithi174.stderr: 93.8% -- replaced with /var/log/ceph/df845a14-c7de-11ee-95b8-87774f69a715/ceph-osd.1.log.gz 2024-02-10T07:05:40.736 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-02-10T07:05:40.736 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m7.821s 2024-02-10T07:05:40.736 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m31.713s 2024-02-10T07:05:40.736 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m1.135s 2024-02-10T07:05:40.736 INFO:tasks.cephadm:Archiving logs... 2024-02-10T07:05:40.736 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/remote/smithi174/log 2024-02-10T07:05:40.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T07:05:45.163 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/remote/smithi184/log 2024-02-10T07:05:45.164 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T07:05:49.035 INFO:tasks.cephadm:Removing cluster... 2024-02-10T07:05:49.035 DEBUG:teuthology.orchestra.run.smithi174:> sudo cephadm rm-cluster --fsid df845a14-c7de-11ee-95b8-87774f69a715 --force 2024-02-10T07:05:49.228 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T07:05:49.520 DEBUG:teuthology.orchestra.run.smithi184:> sudo cephadm rm-cluster --fsid df845a14-c7de-11ee-95b8-87774f69a715 --force 2024-02-10T07:05:49.711 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: df845a14-c7de-11ee-95b8-87774f69a715 2024-02-10T07:05:49.996 INFO:tasks.cephadm:Teardown complete 2024-02-10T07:05:49.996 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-10T07:05:50.006 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-10T07:05:50.006 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-10T07:05:50.010 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-10T07:05:50.087 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-10T07:05:50.096 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-10T07:05:50.096 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:05:50.099 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:05:50.111 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:05:50.113 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:05:50.115 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:05:50.115 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-02-10T07:05:50.115 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +894us[ +849us] +/- 98ms 2024-02-10T07:05:50.115 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -844us[ -889us] +/- 46ms 2024-02-10T07:05:50.115 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +563us[ +563us] +/- 73ms 2024-02-10T07:05:50.115 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:05:50.117 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:05:50.117 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-02-10T07:05:50.117 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +921us[ +894us] +/- 98ms 2024-02-10T07:05:50.117 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -816us[ -843us] +/- 46ms 2024-02-10T07:05:50.117 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +550us[ +550us] +/- 73ms 2024-02-10T07:05:50.117 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:05:50.118 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-10T07:05:50.126 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-10T07:05:50.127 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-10T07:05:50.137 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T07:05:50.237 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T07:05:50.310 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:05:50.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:05:50.341 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:05:50.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:05:50.372 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-10T07:05:50.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-10T07:05:50.401 INFO:teuthology.task.internal:Duration was 2101.369452 seconds 2024-02-10T07:05:50.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-10T07:05:50.410 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-10T07:05:50.410 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T07:05:50.413 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T07:05:50.453 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:05:50.456 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:05:50.778 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-10T07:05:50.778 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-02-10T07:05:50.778 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T07:05:50.837 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-02-10T07:05:50.838 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T07:05:50.867 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-10T07:05:50.868 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T07:05:50.881 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T07:05:50.936 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-10T07:05:50.936 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T07:05:50.949 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T07:05:51.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-10T07:05:51.371 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-10T07:05:51.371 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T07:05:51.398 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T07:05:51.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-10T07:05:51.436 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:05:51.440 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:05:51.463 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-02-10T07:05:51.491 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-02-10T07:05:51.516 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:05:51.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:05:51.533 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:05:51.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:05:51.571 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-10T07:05:51.581 INFO:teuthology.task.internal:Transferring archived files... 2024-02-10T07:05:51.582 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/remote/smithi174 2024-02-10T07:05:51.582 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T07:05:51.646 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156/remote/smithi184 2024-02-10T07:05:51.646 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T07:05:51.687 INFO:teuthology.task.internal:Removing archive directory... 2024-02-10T07:05:51.687 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T07:05:51.689 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T07:05:51.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-10T07:05:51.747 INFO:teuthology.task.internal:Not uploading archives. 2024-02-10T07:05:51.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-10T07:05:51.757 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-10T07:05:51.757 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T07:05:51.760 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T07:05:51.772 INFO:teuthology.orchestra.run.smithi174.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 07:05 /home/ubuntu/cephtest 2024-02-10T07:05:51.789 INFO:teuthology.orchestra.run.smithi184.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 07:05 /home/ubuntu/cephtest 2024-02-10T07:05:51.790 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-10T07:05:51.801 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-10T07:05:51.825 INFO:teuthology.nuke:Checking targets against current locks 2024-02-10T07:05:51.852 DEBUG:teuthology.nuke:shortname: smithi174 2024-02-10T07:05:51.853 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:05:51.874 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:22:26.816383', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtfB6wEcBjgFpkXf65JhQSnrXfX7dab1uY+4Inh9Z3YvQ0l4YzMF9xaTxI+nVJrr3uvjyhLohrrJ9p05IN9NZ8='} 2024-02-10T07:05:51.878 DEBUG:teuthology.nuke:shortname: smithi184 2024-02-10T07:05:51.878 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:05:51.902 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555156', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 06:22:26.818390', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJk03Y/wGdHEm0gebY0RdgrkzCNO+u2t7SRuau3alA2kUuzzhtq6HGfdkuXleBThex9QTSkevET9g3r4soWn2iY='} 2024-02-10T07:05:51.929 INFO:teuthology.orchestra.console.smithi174:Power off 2024-02-10T07:05:51.929 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T07:05:51.953 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2024-02-10T07:05:51.954 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:05:51.971 INFO:teuthology.orchestra.console.smithi184:Power off 2024-02-10T07:05:51.971 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T07:05:51.982 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-02-10T07:05:51.995 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2024-02-10T07:05:51.996 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:05:52.020 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-02-10T07:05:55.984 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:05:56.021 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:05:56.099 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-02-10T07:05:56.134 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-02-10T07:06:00.104 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:06:00.134 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:06:00.215 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-02-10T07:06:00.244 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2024-02-10T07:06:00.316 INFO:teuthology.orchestra.console.smithi174:Power off completed 2024-02-10T07:06:00.345 INFO:teuthology.orchestra.console.smithi184:Power off completed 2024-02-10T07:06:00.449 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} duration: 2101.3694517612457 failure_reason: '"2024-02-10T06:50:00.000246+0000 mon.a (mon.0) 2217 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-02-10T07:06:00.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T07:06:00.538 INFO:teuthology.run:FAIL